Difference between revisions of "Atlas:Analysis Challenge ST"

Un article de lcgwiki.
Jump to: navigation, search
(→‎Test conditions)
(→‎Test conditions)
Ligne 9: Ligne 9:
 
==== Test conditions ====
 
==== Test conditions ====
  
−
* The testing framework is ganga-based. It is currently using LCG backend but it will soon be possible to use PANDA backend as well.
+
* The testing framework is ganga-based. It is currently using LCG backend but it will soon be possible to use PANDA backend as well. Metrics are collected and displayed at
 
* Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes. <br>
 
* Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes. <br>
 
* It uses regular AOD analysis in 14.2.20 with mc08*AOD*e*s*r5 DQ2 inputs<br>
 
* It uses regular AOD analysis in 14.2.20 with mc08*AOD*e*s*r5 DQ2 inputs<br>
Ligne 16: Ligne 16:
 
* Jobs run under DN : /O=GermanGrid/OU=LMU/CN=Johannes_Elmsheuser  
 
* Jobs run under DN : /O=GermanGrid/OU=LMU/CN=Johannes_Elmsheuser  
 
* LAN saturation observed in case of 1 Gb network connection between WN and SE. <br>
 
* LAN saturation observed in case of 1 Gb network connection between WN and SE. <br>
−
 
+
* It is possible for sites to limit the number of jobs sent at a time.
−
* GangaRobot
 
−
DA challenge runs similar to existing analysis functional tests : http://gangarobot.cern.ch/<br>
 
−
See [http://indico.cern.ch/getFile.py/access?contribId=63&sessionId=11&resId=0&materialId=slides&confId=22137 J.Elmsheuser's presentation (Nov. 5 08)]<br>
 
−
<b> Site should pass the GangaRobot test successfully </b> , especially :
 
−
* http://gangarobot.cern.ch/20081119_02/index.html
 
−
* http://gangarobot.cern.ch/20081119_03/index.html
 
−
 
 
−
<b>Participation required at cloud and site level. Any site in the Tiers_of_ATLAS list can participate.</b>
 
−
 
 
−
It is possible for sites to limit the number of jobs sent at a time.  
 
−
DA team is ready to take into account site constraints.
 
−
DA team is open to any metrics
 
  
 
=== Target and metrics ===
 
=== Target and metrics ===

Version du 16:36, 20 janvier 2009

Site Stress Test

Procedure

  • Replication of target datasets accross the cloud
  • Preparation of job
  • Generation n jobs per site (Each job processes 1 dataset)
  • Bulk submission to WMS (1 per site)

Test conditions

  • The testing framework is ganga-based. It is currently using LCG backend but it will soon be possible to use PANDA backend as well. Metrics are collected and displayed at
  • Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes.
  • It uses regular AOD analysis in 14.2.20 with mc08*AOD*e*s*r5 DQ2 inputs
  • Input datasets are read from ATLASMCDISK and outputs are stored on ATLASUSERDISK (no special requirements there). Input data access is the main issue. No problem on data output
  • Required CPUtime : GlueCEPolicyMaxCPUTime >= 1440 (1 day , typical duration : 5 hours)
  • Jobs run under DN : /O=GermanGrid/OU=LMU/CN=Johannes_Elmsheuser
  • LAN saturation observed in case of 1 Gb network connection between WN and SE.
  • It is possible for sites to limit the number of jobs sent at a time.

Target and metrics

  • Nb of jobs : Few hundred up to 1000 jobs/site
  • Rate (evt/s) : up to 15 Hz
  • Success rate (success/failure rate) > 80 %
  • CPU utilization : CPUtime / Walltime > 50 %

Results

See

ST summary

Récupérée de « http://lcg.in2p3.fr/wiki/index.php?title=Atlas:Analysis_Challenge_ST&oldid=4688 »