Difference between revisions of "Atlas:Analysis Challenge ST"
(→[http://lcg.in2p3.fr/wiki/index.php/Atlas:Analysis_Challenge-STsummary ST summary on the FR Cloud]) |
(→More information) |
||
Ligne 29: | Ligne 29: | ||
=== More information === | === More information === | ||
+ | * Latest news from ADC developent meeting : http://indico.cern./conferenceDisplay.py?confId=48239 | ||
+ | **Usage of a prestager (copy in background of needed files on WN while processing) improve on most sites CPU/Walltime ratio (up to >90%) and Nb of events > 20Hz. However.. lcg-cp is used for copy... | ||
+ | **some tests were performed on the whole LCG cloud like this one | ||
+ | http://gangarobot.cern.ch/st/test_105/ | ||
* Results of analysis challenge performed on [http://indico.cern.ch/getFile.py/access?contribId=128&sessionId=8&resId=0&materialId=slides&confId=22137 IT Cloud] | * Results of analysis challenge performed on [http://indico.cern.ch/getFile.py/access?contribId=128&sessionId=8&resId=0&materialId=slides&confId=22137 IT Cloud] | ||
* Results of [http://indico.cern.ch/getFile.py/access?contribId=129&sessionId=8&resId=0&materialId=slides&confId=22137 DE Cloud] | * Results of [http://indico.cern.ch/getFile.py/access?contribId=129&sessionId=8&resId=0&materialId=slides&confId=22137 DE Cloud] | ||
===[http://lcg.in2p3.fr/wiki/index.php/Atlas:Analysis_Challenge-STsummary FR-Cloud ST summary (12/08)]=== | ===[http://lcg.in2p3.fr/wiki/index.php/Atlas:Analysis_Challenge-STsummary FR-Cloud ST summary (12/08)]=== |
Version du 15:47, 20 janvier 2009
Site Stress Test
Sommaire
Procedure
- Replication of target datasets accross the cloud
- Preparation of job
- Generation n jobs per site (Each job processes 1 dataset)
- Bulk submission to WMS (1 per site)
Test conditions
- The testing framework is ganga-based. It is currently using LCG backend but it will soon be possible to use PANDA backend as well. Metrics are collected and displayed at http://gangarobot.cern.ch/st/
- Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes.
- It uses regular AOD analysis in 14.2.20 with mc08*AOD*e*s*r5 DQ2 inputs
- Input datasets are read from ATLASMCDISK and outputs are stored on ATLASUSERDISK (no special requirements there). Input data access is the main issue. No problem on data output
- Required CPUtime : GlueCEPolicyMaxCPUTime >= 1440 (1 day , typical duration : 5 hours)
- Jobs run under DN : /O=GermanGrid/OU=LMU/CN=Johannes_Elmsheuser
- LAN saturation observed in case of 1 Gb network connection between WN and SE.
- It is possible for sites to limit the number of jobs sent at a time.
- Test duration : 48 hours
Target and metrics
- Nb of jobs : Few hundred up to 1000 jobs/site
- Rate (evt/s) : up to 15 Hz
- Success rate (success/failure rate) > 80 %
- CPU utilization : CPUtime / Walltime > 50 %
Results and Monitoring
More information
- Latest news from ADC developent meeting : http://indico.cern./conferenceDisplay.py?confId=48239
- Usage of a prestager (copy in background of needed files on WN while processing) improve on most sites CPU/Walltime ratio (up to >90%) and Nb of events > 20Hz. However.. lcg-cp is used for copy...
- some tests were performed on the whole LCG cloud like this one
http://gangarobot.cern.ch/st/test_105/