Difference between revisions of "Atlas:Analysis Challenge"

Un article de lcgwiki.
Jump to: navigation, search
(Phase 1 : Site stress test oraganized by ATLAS and run centrally in a controlled manner (2 days))
(Phase 1 : Site stress test oraganized by ATLAS and run centrally in a controlled manner (2 days))
Ligne 56: Ligne 56:
 
   <span style="color:orange;">Problem tracked</span>
 
   <span style="color:orange;">Problem tracked</span>
 
* <b>Scheduled Tests </b>
 
* <b>Scheduled Tests </b>
Test #61:
+
Test #61: http://gangarobot.cern.ch/st/test_61/
 
  Start Time: 2008-12-08 10:00:00
 
  Start Time: 2008-12-08 10:00:00
 
  End Time: 2008-12-10 10:00:00
 
  End Time: 2008-12-10 10:00:00
Ligne 63: Ligne 63:
 
  Input Type: DQ2_LOCAL
 
  Input Type: DQ2_LOCAL
 
  Sites: IN2P3-LPC_MCDISK GRIF-LPNHE_MCDISK TOKYO-LCG2_MCDISK IN2P3-CPPM_MCDISK
 
  Sites: IN2P3-LPC_MCDISK GRIF-LPNHE_MCDISK TOKYO-LCG2_MCDISK IN2P3-CPPM_MCDISK
  Input DS Patterns: mc08.*Wmunu*.recon.AOD.e*_s*_r5*tid* mc08.*Zprime_mumu*.recon.AOD.e*_s*_r5*tid* mc08.*Zmumu*.recon.AOD.e*_s*_r5*tid* mc08.*T1_McAtNlo*.recon.AOD.e*_s*_r5*tid* mc08.*H*zz4l*.recon.AOD.e*_s*_r5*tid* mc08.*.recon.AOD.e*_s*_r5*tid*
+
  Input DS Patterns: mc08.*Wmunu*.recon.AOD.e*_s*_r5*tid*  
 +
                    mc08.*Zprime_mumu*.recon.AOD.e*_s*_r5*tid*
 +
                    mc08.*Zmumu*.recon.AOD.e*_s*_r5*tid*
 +
                    mc08.*T1_McAtNlo*.recon.AOD.e*_s*_r5*tid*                
 +
                    mc08.*H*zz4l*.recon.AOD.e*_s*_r5*tid* mc08.*.recon.AOD.e*_s*_r5*tid*
  
Test #62:
+
Test #62: http://gangarobot.cern.ch/st/test_62/
 
  Start Time: 2008-12-08 10:00:00
 
  Start Time: 2008-12-08 10:00:00
 
  End Time: 2008-12-10 10:00:00
 
  End Time: 2008-12-10 10:00:00
Ligne 72: Ligne 76:
 
  Input Type: DQ2_LOCAL
 
  Input Type: DQ2_LOCAL
 
  Sites: TOKYO-LCG2_MCDISK
 
  Sites: TOKYO-LCG2_MCDISK
  Input DS Patterns: mc08.*Wmunu*.recon.AOD.e*_s*_r5*tid* mc08.*Zprime_mumu*.recon.AOD.e*_s*_r5*tid* mc08.*Zmumu*.recon.AOD.e*_s*_r5*tid* mc08.*T1_McAtNlo*.recon.AOD.e*_s*_r5*tid* mc08.*H*zz4l*.recon.AOD.e*_s*_r5*tid* mc08.*.recon.AOD.e*_s*_r5*tid*
+
  Input DS Patterns: mc08.*Wmunu*.recon.AOD.e*_s*_r5*tid*  
 +
                    mc08.*Zprime_mumu*.recon.AOD.e*_s*_r5*tid*  
 +
                    mc08.*Zmumu*.recon.AOD.e*_s*_r5*tid*  
 +
                    mc08.*T1_McAtNlo*.recon.AOD.e*_s*_r5*tid*  
 +
                    mc08.*H*zz4l*.recon.AOD.e*_s*_r5*tid* mc08.*.recon.AOD.e*_s*_r5*tid*
  
 
===== Phase 2 : Pathena Analysis Challenge  =====
 
===== Phase 2 : Pathena Analysis Challenge  =====

Version du 06:27, 8 décembre 2008

02/12/08 : E.Lançon, F.Chollet (Thanks to Cédric Serfon)

Information & Contact

Mailing list ATLAS-LCG-OP-L@in2p3.fr

Goals

  • measure "real" analysis job efficiency and turn around on several sites of a given cloud
  • measure data access performance
  • check load balancing between different users and different analysis tools (Ganga vs pAthena)
  • check load balancing between analysis and MC production

Required services @ T1

  • LFC catalog : lfc-prod.in2p3.fr
  • ATLAS Disk space : ATLASUSERDISK on T1 SE (fail-over for outputs in case of problems with T2 disk storage)

First exercise on the FR Cloud (>= December 8th )

Phase 1 : Site stress test oraganized by ATLAS and run centrally in a controlled manner (2 days)

DA challenges have been performed on IT and DE clouds in october 08. Proposition has been made to extend this cloud-by cloud challenge to the FR Cloud. See ATLAS coordination DA challenge meeting (Nov. 20)

First exercise will help to identify breaking points and bottlenecks. It is limited in time (a few days) and requires careful attention of site administrators during that period,in particular network (internal & external), disk, cpu monitoring. This first try (Stress tests) can be run centrally in a controlled manner. The testing framework is ganga-based.

Participation required at cloud and site level. Any site in the Tiers_of_ATLAS list can participate. ATLAS coordination (Dan van der Ster and Johannes Elmsheuser) needs to know which sites to be tested and when.

It is possible for sites to limit the number of jobs sent at a time. 
DA team is ready to take into account site constraints.
DA team is open to any metrics

GlueCEPolicyMaxCPUTime >= 1440 ( 1 day)

Start Time: 2008-11-28 13:00:00
End Time: 2008-11-28 21:00:00 (test interrupted after 8 hours). Some jobs remain in "running" state.
Job status : c (Completed) / r (Running) / f (Failed)
o|e : links to stdout (o) et stderr (e)

Problems seen : 
- Missing ATLAS release 14.2.20 @ CPPM 
  02/12/08 [DONE] validate-prod of 14.2.20 at marce01.in2p3.fr (IN2P3-CPPM)
- Mapping CE-SE@GRIF-LAL and GRIF-SACLAY : Publication of multiple CloseSEs 
  by LAL and IRFU cf https://savannah.cern.ch/bugs/index.php?44824
- LFC bulk reading...End of LFC bulk reading...ERROR: Dataset(s)...is/are
  empty at GRIF-SACLAY_MCDISK / LAL_MCDISK 
 (could be related to same problem)
  Problem tracked
- ERROR@RO7-NIPNE : failed jobs with Athena errors 
  NON FATAL ERRORS (completed jobs) @RO7-NIPNE, RO2-NIPNE : related to
  ATLASUSERDISK space token configuration
  ERROR during execution of lcg-cr --vo atlas -s ATLASUSERDISK
  ERROR: file not saved to RO-07-NIPNE_USERDISK in attempt number 3 ...
  ERROR: file not saved to RO-07-NIPNE_USERDISK - using now IN2P3-CC_USERDISK
  Problem tracked
  • Scheduled Tests

Test #61: http://gangarobot.cern.ch/st/test_61/

Start Time: 2008-12-08 10:00:00
End Time: 2008-12-10 10:00:00
Output Dataset: user08.JohannesElmsheuser.ganga.sitetest.FR.081208.<sitename>
Max Jobs Per Site: 300
Input Type: DQ2_LOCAL
Sites: IN2P3-LPC_MCDISK GRIF-LPNHE_MCDISK TOKYO-LCG2_MCDISK IN2P3-CPPM_MCDISK
Input DS Patterns: mc08.*Wmunu*.recon.AOD.e*_s*_r5*tid* 
                   mc08.*Zprime_mumu*.recon.AOD.e*_s*_r5*tid*  
                   mc08.*Zmumu*.recon.AOD.e*_s*_r5*tid*  
                   mc08.*T1_McAtNlo*.recon.AOD.e*_s*_r5*tid*                  
                   mc08.*H*zz4l*.recon.AOD.e*_s*_r5*tid* mc08.*.recon.AOD.e*_s*_r5*tid*

Test #62: http://gangarobot.cern.ch/st/test_62/

Start Time: 2008-12-08 10:00:00
End Time: 2008-12-10 10:00:00
Output Dataset: user08.JohannesElmsheuser.ganga.sitetest.FR.081208.<sitename>
Max Jobs Per Site: 300
Input Type: DQ2_LOCAL
Sites: TOKYO-LCG2_MCDISK
Input DS Patterns: mc08.*Wmunu*.recon.AOD.e*_s*_r5*tid* 
                   mc08.*Zprime_mumu*.recon.AOD.e*_s*_r5*tid* 
                   mc08.*Zmumu*.recon.AOD.e*_s*_r5*tid* 
                   mc08.*T1_McAtNlo*.recon.AOD.e*_s*_r5*tid* 
                   mc08.*H*zz4l*.recon.AOD.e*_s*_r5*tid* mc08.*.recon.AOD.e*_s*_r5*tid*
Phase 2 : Pathena Analysis Challenge
  • Data Analysis exercice open to physicists with their favorite application
  • Physicists involved : Julien Donini, Arnaud Lucotte, Bertrand Brelier, Eric Lançon, LAL ?, LPNHE ?

Planning

  • Dec 8 : stop of MC production
  • Dec. 8-9: 1rst round with Tokyo, CPPM, LPC (LAN limited to 1Gbps), GRIF-LPNHE
  • Dec 17 : restart of MC production
  • Dec 14 : stop of MC production
  • Dec. 15-16 : 2nd round with LAPP, CC-IN2P3-T2 (to be confirmed), Tokyo, CPPM, LPC, possibly GRIF (SACLAY, IRFU, LPNHE), RO-07 and RO-02
  • Dec 17 : restart of MC production
  • Dec 17 : Beginning of Analysis Challenge (Phase 2)

Target and metrics

  • Nb of events : Few hundred up to 1000 jobs/site
  • Rate (evt/s) : up to 15 Hz
  • Efficiency (success/failure rate) : 80 %
  • CPU utilization : CPUtime / Walltime > 50 %