Atlas:Analysis Challenge ST - Revision history

Chollet at 11:49, 29 janvier 2009

2009-01-29T11:49:04Z

← Version précédente		Version du 13:49, 29 janvier 2009
Ligne 1:		Ligne 1:
	<b> Site Stress Test </b>		<b> Site Stress Test </b>
			--[[User:Chollet\|Chollet]] 12:49, 29 janvier 2009 (CET)
	==== Procedure ====		==== Procedure ====
	* Replication of target datasets accross the cloud		* Replication of target datasets accross the cloud

Chollet: /* More information */

2009-01-29T11:48:49Z

More information

← Version précédente		Version du 13:48, 29 janvier 2009
Ligne 40:		Ligne 40:
	=== Results and Monitoring ===		=== Results and Monitoring ===
	* See http://gangarobot.cern.ch/st/		* See http://gangarobot.cern.ch/st/

	~~=== More information ===~~
	* Latest news from ADC developent meeting : http://indico.cern./conferenceDisplay.py?confId=48239
	**Usage of a prestager (copy in background of needed files on WN while processing) improve on most sites CPU/Walltime ratio (up to >90%) and Nb of events > 20Hz. However.. lcg-cp is used for copy...
	**some tests were performed on the whole LCG cloud like this one
	~~http://gangarobot.cern.ch/st/test_105/~~
	* Results of analysis challenge performed on [http://indico.cern.ch/getFile.py/access?contribId=128&sessionId=8&resId=0&materialId=slides&confId=22137 IT Cloud]
	* Results of [http://indico.cern.ch/getFile.py/access?contribId=129&sessionId=8&resId=0&materialId=slides&confId=22137 DE Cloud]

Chollet at 11:47, 29 janvier 2009

2009-01-29T11:47:29Z

← Version précédente		Version du 13:47, 29 janvier 2009
Ligne 27:		Ligne 27:

	=== File access modes used by Ganga Jobs ===		=== File access modes used by Ganga Jobs ===

	Two access modes may be used during ST tests.		Two access modes may be used during ST tests.
	* '''(j.inputdata.type='DQ2_LOCAL')''' : Currently if users are submitting Ganga jobs with the LCG backend, the posix I/O access is the default method to access files		* '''(j.inputdata.type='DQ2_LOCAL')''' : Currently if users are submitting Ganga jobs with the LCG backend, the posix I/O access is the default method to access files

Chollet at 11:46, 29 janvier 2009

2009-01-29T11:46:29Z

← Version précédente		Version du 13:46, 29 janvier 2009
Ligne 25:		Ligne 25:
	* It is possible for sites to limit the number of jobs sent at a time.		* It is possible for sites to limit the number of jobs sent at a time.
	* Test duration : 48 hours		* Test duration : 48 hours

			=== File access modes used by Ganga Jobs ===
			Two access modes may be used during ST tests.
			* '''(j.inputdata.type='DQ2_LOCAL')''' : Currently if users are submitting Ganga jobs with the LCG backend, the posix I/O access is the default method to access files
			* '''(j.inputdata.type='FILE_STAGER')''' : One of the alternative access mode used by ST tests is the FileStager mode, which copies the input files in a background thread of the athena event loop usinglcg-cp from the local SE to the worker node tmp area. This mode still needs a bit of improvements to gain full stability, but as demonstrated good performance during ST at some (but not all) sites.

	=== Target and metrics ===		=== Target and metrics ===
Ligne 42:		Ligne 47:
	* Results of analysis challenge performed on [http://indico.cern.ch/getFile.py/access?contribId=128&sessionId=8&resId=0&materialId=slides&confId=22137 IT Cloud]		* Results of analysis challenge performed on [http://indico.cern.ch/getFile.py/access?contribId=128&sessionId=8&resId=0&materialId=slides&confId=22137 IT Cloud]
	* Results of [http://indico.cern.ch/getFile.py/access?contribId=129&sessionId=8&resId=0&materialId=slides&confId=22137 DE Cloud]		* Results of [http://indico.cern.ch/getFile.py/access?contribId=129&sessionId=8&resId=0&materialId=slides&confId=22137 DE Cloud]

	~~===[http://lcg.in2p3.fr/wiki/index.php/Atlas:Analysis_Challenge-STsummary FR-Cloud ST summary (12/08)]===~~

Chollet: /* Test conditions */

2009-01-20T14:57:47Z

Test conditions

← Version précédente		Version du 16:57, 20 janvier 2009
Ligne 18:		Ligne 18:
	mc08.T1_McAtNlo.recon.AOD.e_s_r5tid		mc08.T1_McAtNlo.recon.AOD.e_s_r5tid
	mc08.Hzz4l.recon.AOD.e_s_r5tid*		mc08.Hzz4l.recon.AOD.e_s_r5tid*

	mc08..recon.AOD.e_s_r5tid* (this pattern includes all the previous ones)		mc08..recon.AOD.e_s_r5tid* (this pattern includes all the previous ones)
	* Input datasets are read from ATLASMCDISK and outputs are stored on ATLASUSERDISK (no special requirements there). Input data access is the main issue. No problem on data output <br>		* Input datasets are read from ATLASMCDISK and outputs are stored on ATLASUSERDISK (no special requirements there). Input data access is the main issue. No problem on data output <br>

Chollet: /* Test conditions */

2009-01-20T14:57:34Z

Test conditions

← Version précédente		Version du 16:57, 20 janvier 2009
Ligne 18:		Ligne 18:
	mc08.T1_McAtNlo.recon.AOD.e_s_r5tid		mc08.T1_McAtNlo.recon.AOD.e_s_r5tid
	mc08.Hzz4l.recon.AOD.e_s_r5tid*		mc08.Hzz4l.recon.AOD.e_s_r5tid*
	mc08..recon.AOD.e_s_r5tid*
			mc08..recon.AOD.e_s_r5tid* (this pattern includes all the previous ones)
	* Input datasets are read from ATLASMCDISK and outputs are stored on ATLASUSERDISK (no special requirements there). Input data access is the main issue. No problem on data output <br>		* Input datasets are read from ATLASMCDISK and outputs are stored on ATLASUSERDISK (no special requirements there). Input data access is the main issue. No problem on data output <br>
	* Required CPUtime : GlueCEPolicyMaxCPUTime >= 1440 (1 day , typical duration : 5 hours)		* Required CPUtime : GlueCEPolicyMaxCPUTime >= 1440 (1 day , typical duration : 5 hours)

Chollet: /* Test conditions */

2009-01-20T14:56:56Z

Test conditions

← Version précédente		Version du 16:56, 20 janvier 2009
Ligne 12:		Ligne 12:
	* Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes. <br>		* Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes. <br>
	* ATLAS software release 14.2.20		* ATLAS software release 14.2.20
	* Input DS Patterns used (first one is the preferred one):		* Input DS Patterns used (first one is the preferred one for muon analysis):
	mc08.Wmunu.recon.AOD.e_s_r5tid		mc08.Wmunu.recon.AOD.e_s_r5tid
	mc08.Zprime_mumu.recon.AOD.e_s_r5tid		mc08.Zprime_mumu.recon.AOD.e_s_r5tid

Chollet: /* Test conditions */

2009-01-20T14:56:28Z

Test conditions

← Version précédente		Version du 16:56, 20 janvier 2009
Ligne 11:		Ligne 11:
	* The testing framework is ganga-based. It is currently using LCG backend but it will soon be possible to use PANDA backend as well. Metrics are collected and displayed at http://gangarobot.cern.ch/st/		* The testing framework is ganga-based. It is currently using LCG backend but it will soon be possible to use PANDA backend as well. Metrics are collected and displayed at http://gangarobot.cern.ch/st/
	* Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes. <br>		* Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes. <br>
	* ~~It uses regular AOD analysis and~~ ATLAS software release 14.2.20		* ATLAS software release 14.2.20
	* Input DS Patterns used :		* Input DS Patterns used (first one is the preferred one):
	mc08.Wmunu.recon.AOD.e_s_r5tid		mc08.Wmunu.recon.AOD.e_s_r5tid
	mc08.Zprime_mumu.recon.AOD.e_s_r5tid		mc08.Zprime_mumu.recon.AOD.e_s_r5tid

Chollet: /* Test conditions */

2009-01-20T14:55:33Z

Test conditions

← Version précédente		Version du 16:55, 20 janvier 2009
Ligne 11:		Ligne 11:
	* The testing framework is ganga-based. It is currently using LCG backend but it will soon be possible to use PANDA backend as well. Metrics are collected and displayed at http://gangarobot.cern.ch/st/		* The testing framework is ganga-based. It is currently using LCG backend but it will soon be possible to use PANDA backend as well. Metrics are collected and displayed at http://gangarobot.cern.ch/st/
	* Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes. <br>		* Both POSIX I/O and "copy mode" may be used allowing performances comparaison of the 2 modes. <br>
	* It uses regular AOD analysis in 14.2.20 ~~with~~ mc08AODes~~r5 DQ2 inputs<br>~~		* It uses regular AOD analysis and ATLAS software release 14.2.20
			* Input DS Patterns used :
			mc08.Wmunu.recon.AOD.e_s_r5tid
			mc08.Zprime_mumu.recon.AOD.e_s_r5tid
			mc08.Zmumu.recon.AOD.e_s_r5tid
			mc08.T1_McAtNlo.recon.AOD.e_s_r5tid
			mc08.Hzz4l.recon.AOD.e_s_r5tid*
			mc08..recon.AOD.e_s_r5tid*
	* Input datasets are read from ATLASMCDISK and outputs are stored on ATLASUSERDISK (no special requirements there). Input data access is the main issue. No problem on data output <br>		* Input datasets are read from ATLASMCDISK and outputs are stored on ATLASUSERDISK (no special requirements there). Input data access is the main issue. No problem on data output <br>
	* Required CPUtime : GlueCEPolicyMaxCPUTime >= 1440 (1 day , typical duration : 5 hours)		* Required CPUtime : GlueCEPolicyMaxCPUTime >= 1440 (1 day , typical duration : 5 hours)

Chollet: /* More information */

2009-01-20T14:47:38Z

More information

← Version précédente		Version du 16:47, 20 janvier 2009
Ligne 29:		Ligne 29:

	=== More information ===		=== More information ===
			* Latest news from ADC developent meeting : http://indico.cern./conferenceDisplay.py?confId=48239
			**Usage of a prestager (copy in background of needed files on WN while processing) improve on most sites CPU/Walltime ratio (up to >90%) and Nb of events > 20Hz. However.. lcg-cp is used for copy...
			**some tests were performed on the whole LCG cloud like this one
			http://gangarobot.cern.ch/st/test_105/
	* Results of analysis challenge performed on [http://indico.cern.ch/getFile.py/access?contribId=128&sessionId=8&resId=0&materialId=slides&confId=22137 IT Cloud]		* Results of analysis challenge performed on [http://indico.cern.ch/getFile.py/access?contribId=128&sessionId=8&resId=0&materialId=slides&confId=22137 IT Cloud]
	* Results of [http://indico.cern.ch/getFile.py/access?contribId=129&sessionId=8&resId=0&materialId=slides&confId=22137 DE Cloud]		* Results of [http://indico.cern.ch/getFile.py/access?contribId=129&sessionId=8&resId=0&materialId=slides&confId=22137 DE Cloud]

	===[http://lcg.in2p3.fr/wiki/index.php/Atlas:Analysis_Challenge-STsummary FR-Cloud ST summary (12/08)]===		===[http://lcg.in2p3.fr/wiki/index.php/Atlas:Analysis_Challenge-STsummary FR-Cloud ST summary (12/08)]===