Difference between revisions of "Atlas:Analysis Challenge"

Un article de lcgwiki.
Jump to: navigation, search
(Results)
(Results)
Ligne 36: Ligne 36:
 
See  
 
See  
 
* ATLAS Twiki page : https://twiki.cern.ch/twiki/bin/view/Main/GangaSiteTests  
 
* ATLAS Twiki page : https://twiki.cern.ch/twiki/bin/view/Main/GangaSiteTests  
* Results of analysis challenge performed on [IT Cloud | http://indico.cern.ch/getFile.py/access?contribId=128&sessionId=8&resId=0&materialId=slides&confId=22137]  
+
* Results of analysis challenge performed on [http://indico.cern.ch/getFile.py/access?contribId=128&sessionId=8&resId=0&materialId=slides&confId=22137 IT Cloud]  
 
* Results of [DE Cloud | http://indico.cern.ch/getFile.py/access?contribId=129&sessionId=8&resId=0&materialId=slides&confId=22137]
 
* Results of [DE Cloud | http://indico.cern.ch/getFile.py/access?contribId=129&sessionId=8&resId=0&materialId=slides&confId=22137]

Version du 16:05, 21 novembre 2008

ATLAS Analysis challenge

Goals

  • measure "real" analysis job efficiency and turn around on several sites of a given cloud
  • measure data access performance
  • check load balancing between different users and different analysis tools (Ganga vs pAthena)
  • check load balancing between analysis and MC production

First exercise on the FR Cloud in december 08 (proposition)

DA challenges have been performed on IT and DE clouds in october 08. Proposition has been made to extend this cloud-by cloud challenge to the FR Cloud. See ATLAS coordination DA challenge meeting (Nov. 20)

First exercise will help to identify breaking points and bottlenecks. It is limited in time (a few days) and requires careful attention of site administrators during that period,in particular network (internal & external), disk, cpu monitoring.

This first try can be run centrally. ATLAS coordination (Dan van der Ster and Johannes Elmsheuser) needs to know which sites to be tested and when. Any site in the Tiers_of_ATLAS list can participate.

Procedure

  • Replication of target datasets accross the cloud
  • Preparation of job
  • Generation n jobs per site (Each job processes 1 dataset)
  • Bulk submission to WMS (1 per site)

Testing framework and conditions

The testing framework is ganga-based. It is currently using LCG backend but it will soon be possible to use PANDA backend as well. Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes. It uses real analysis code (typ. AOD muon analysis)

It is possible for sites to limit the number of jobs sent at a time. 
DA team is ready to take into account site constraints or limitations.
DA team is open to any metrics

2009 plans

Results

See