Difference between revisions of "Atlas:SC4"

Un article de lcgwiki.
Jump to: navigation, search
(Daily news)
(‎)
 
(67 intermediate revisions by 19 users not shown)
Ligne 2: Ligne 2:
 
[https://uimon.cern.ch/twiki/bin/view/Atlas/ATLASServiceChallenges Twiki page : SC4 ATLAS]
 
[https://uimon.cern.ch/twiki/bin/view/Atlas/ATLASServiceChallenges Twiki page : SC4 ATLAS]
  
[http://lcg2.in2p3.fr/wiki/images/Sc4-juin06.txt Compte rendu de la réunion SC4 ATLAS au CERN du 9 Juin (S.Jézéquel) (written in french) ]
+
[http://lcg2.in2p3.fr/wiki/images/Sc4-juin06.txt Compte rendu de la réunion SC4 ATLAS au CERN du 9 Juin (S.Jézéquel, G. Rahal) (written in french) ]
  
 
* T0 Role(CERN)
 
* T0 Role(CERN)
** Produce dummy files with 1 to 2 GB size(RAW, ESD et AOD)
+
** Produce dummy files with 1 to 2 GB size(RAW, ESD et AOD) (see [https://uimon.cern.ch/twiki/bin/view/Atlas/AtlasTierZero T0 Twiki])
 
** Initiate T0->T1 transfers
 
** Initiate T0->T1 transfers
 
** FTS server sents files to Lyon choosing between 'TAPE' (RAW 43,2 Mo/s) or 'DISK' (ESD,AOD 23+20 Mo/s) areas
 
** FTS server sents files to Lyon choosing between 'TAPE' (RAW 43,2 Mo/s) or 'DISK' (ESD,AOD 23+20 Mo/s) areas
 
* T1 Role (CCIN2P3)
 
* T1 Role (CCIN2P3)
** Get files from T0 (dedicated dcache area)
+
** Get files from T0 (dedicated dcache area: L. Schwarz)
 +
** Provides LFC (lfc-atlas.in2p3.fr) and FTS service (cclcgftsli01.in2p3.fr) (D. Bouvet)
 
** Send all AODs to each T2 (20 Mo/s) using Lyon FTS server
 
** Send all AODs to each T2 (20 Mo/s) using Lyon FTS server
 
** Regurlarly cleanup files
 
** Regurlarly cleanup files
* T2 Role (BEIJING, LAL, LPC, TOKYO)
+
* T2 Role (BEIJING, LAL, LAPP, LPC, LPNHE, SACLAY, TOKYO)
** Get files from T1 (Lyon)
+
** Get files from T1 (Lyon). Files on the T2 are written in /home/atlas/sc4tier0/...
 
** Clean-up the files  (?)
 
** Clean-up the files  (?)
 
* Other roles
 
* Other roles
** ATLAS (S. Jézéquel) : Initiate T1->T2 transfers
+
** ATLAS (S. Jézéquel, G. Rahal) : Initiate T1->T2 transfers
  
 
== Information from DDM monitoring ==
 
== Information from DDM monitoring ==
 +
* [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/ Main DDM monitoring page]
  
<span style="color:#FF0000;">According to DDM team, the monitoring is too slow to display the correct information.
+
* http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/rrd/plots/stack4h.png http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/rrd/plots/stack4ht2.png
Untill this is solved, only FTS transfer rates provided by Gridview are displayed</span>:
 
[http://gridview.cern.ch/GRIDVIEW/graphs.php?GraphName=atlas&ThruputDataOption=&SrcSite=&DestSite=&DurationOption=&StartDay=&StartMonth=&StartYear=&EndDay=&EndMonth=&EndYear=&HostType=&GraphFor=VO  Atlas FTS transfer rate]
 
[http://gridview.cern.ch/GRIDVIEW/graphs.php?GraphName=IN2PCC&ThruputDataOption=&SrcSite=&DestSite=&DurationOption=&StartDay=&StartMonth=&StartYear=&EndDay=&EndMonth=&EndYear=&HostType=&GraphFor= Atlas CERN->Lyon FTS transfer rate]
 
  
* [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/siteplots.php?site=LYON Transfer rates for LYON (validated files by DDM)]
+
----
* [http://atldq02.cern.ch:8000/dq2/new_style_site_monitor/sites Detailed information on each site ]
+
 
* [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/rrd/plots/stackday.png]
+
* http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/rrd/plots/stackday.png
* [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/rrd/plots/stack4h.png]
+
* http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/rrd/plots/stackdayLYON.png
 +
* http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/rrd/plots/stackdayt2.png
 +
 
 +
----
 +
 
 +
* http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/rrd/plots/stackweek.png
 +
* http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/rrd/plots/stackweekLYON.png
 +
* http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/rrd/plots/stackweekt2.png 
 +
 
 +
--
 +
 
 +
* Transfer rates for LYON and associated T2s (validated files by DDM):
 +
** [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/siteplots.php?site=LYON LYON]
 +
** [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/siteplotst2.php?site=BEIJING BEIJING]
 +
** [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/siteplotst2.php?site=LAL LAL]
 +
** [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/siteplotst2.php?site=LAPP LAPP]
 +
** [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/siteplotst2.php?site=LPC LPC]
 +
** [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/siteplotst2.php?site=LPNHE LPNHE]
 +
** [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/siteplotst2.php?site=SACLAY SACLAY]
 +
** [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/siteplotst2.php?site=TOKYO TOKYO]
 +
 
 +
* [http://atldq02.cern.ch:8000/dq2/site_monitor/sites Detailed information on each site ]
 +
 
 +
* [http://atlas-ddm-monitoring.web.cern.ch/atlas-ddm-monitoring/sitestates.php?site=LYON File transfer state in LYON ]
 +
 
 +
== Information from FTS monitoring ==
 +
 
 +
*[http://gridview.cern.ch/GRIDVIEW/graphs.php?GraphName=atlas&ThruputDataOption=&SrcSite=&DestSite=&DurationOption=&StartDay=&StartMonth=&StartYear=&EndDay=&EndMonth=&EndYear=&HostType=&GraphFor=VO    Atlas FTS transfer rate]
 +
 
 +
* [http://gridview.cern.ch/GRIDVIEW/graphs.php?GraphName=IN2PCC&ThruputDataOption=&SrcSite=&DestSite=&DurationOption=&StartDay=&StartMonth=&StartYear=&EndDay=&EndMonth=&EndYear=&HostType=&GraphFor= CERN->Lyon FTS transfer rate]
 +
 
 +
*T1->T2 :
 +
** 15 concurrent files and 10 streams for LYON-TOKYO
 +
** 5 concurrent files and 5 streams for LYON-BEIJING (SE not enough powerful for 15/15  )
 +
** 10 concurrent files and 10 streams for LYON-French T2s (LAL, LPNHE, LPC, SACLAY) except for LAPP (5 concurrent files and 1 stream)
 +
 
 +
== Information from dCache monitoring (provided by Lyon) ==
 +
 
 +
*[http://cctools.in2p3.fr/dcache/transfers/atlas-sc4.html Monitoring of dcache for SC4 areas]
 +
* LYONDISK : 25 gridftp concurrent access maximum
 +
* LYONTAPE : 10 gridft concurrent access maximum
 +
 
 +
== VOBOX Configuration ==
 +
 
 +
* 4 processors 3 GHz
 +
* 4 GB of memory ( 2 GB dedicated to SWAP)
 +
* The monitoring of the Vobox daily, weekly and monthly can be found [http://atlas-france.in2p3.fr/Activites/Informatique/OutilsCC/VO-cclcgatlas Here]
 +
 
 +
== Disk space availability ==
 +
 
 +
* [http://gridice2.cnaf.infn.it:50080/gridice/site/site_details.php?siteName=BEIJING-LCG2&visibility=SE BEIJING]
 +
* [http://gridice2.cnaf.infn.it:50080/gridice/site/site_details.php?siteName=CPPM-LCG2&visibility=SE CPPM]
 +
* [http://gridice2.cnaf.infn.it:50080/gridice/site/site_details.php?siteName=GRIF&visibility=SE GRIF]
 +
* [http://gridice2.cnaf.infn.it:50080/gridice/site/site_details.php?siteName=IN2P3-LPC&visibility=SE LPC]
 +
* [http://gridice2.cnaf.infn.it:50080/gridice/site/site_details.php?siteName=IN2P3-LAPP&visibility=SE LAPP]
 +
* [http://gridice2.cnaf.infn.it:50080/gridice/site/site_details.php?siteName=TOKYO-LCG2&visibility=SE TOKYO]
 +
* [http://gridice2.cnaf.infn.it:50080/gridice/site/site_details.php?siteName=IN2P3-CC&visibility=SE LYON]
  
 
== Daily news ==
 
== Daily news ==
  
 +
* [https://uimon.cern.ch/twiki/bin/view/Atlas/DDMSc4#Daily_log SC4 Data Managment Daily log]
 +
 +
 +
* SC4 transfers from LYON to T2s:
 +
** Simultaneous transfers to  the 7 T2 sites for more than 24 hours; it reached  more than 25MB/s
 +
**[[Image:StackdayLYON-27-7.png]]
  
* 20 June 2006: Mail from Miguel Branco (DDM responsible)
+
== Post-mortem DDM meeting ==
 +
* [http://indico.cern.ch/conferenceDisplay.py?confId=4959 Presentation at CERN (M. Branco) showing the results of the SC4 tests.] (1 August 2006)
  
<span style="color:#663300;">Today we started deploying DQ2 on the remaining T1 sites (not all
+
== [[Atlas | Page principale du Twiki ATLAS]] ==
sites still available).<br/>
 
Attached is the result of a (nice) ramp up, easily beating SC3's
 
record (on the 1st day of export of SC4) peaking at ~ 270 MB/s. Each
 
'step' in the graph is an additional T1 being added to the export.<br/>
 
Dataset subscriptions are now slowing down and will resume tomorrow.
 
Our DQ2 monitoring has been turned off and we expect to have it back
 
tomorrow! Still a long way to go until we have a reasonable
 
understanding of the limiting factors..
 
</span>
 
[[Image:Exemple.jpg]]
 

Latest revision as of 14:20, 1 octobre 2006

Bienvenue sur la page Atlas SC4 LCG-France

Welcome to the LCG-France Atlas SC4 page

Twiki page : SC4 ATLAS

Compte rendu de la réunion SC4 ATLAS au CERN du 9 Juin (S.Jézéquel, G. Rahal) (written in french)

  • T0 Role(CERN)
    • Produce dummy files with 1 to 2 GB size(RAW, ESD et AOD) (see T0 Twiki)
    • Initiate T0->T1 transfers
    • FTS server sents files to Lyon choosing between 'TAPE' (RAW 43,2 Mo/s) or 'DISK' (ESD,AOD 23+20 Mo/s) areas
  • T1 Role (CCIN2P3)
    • Get files from T0 (dedicated dcache area: L. Schwarz)
    • Provides LFC (lfc-atlas.in2p3.fr) and FTS service (cclcgftsli01.in2p3.fr) (D. Bouvet)
    • Send all AODs to each T2 (20 Mo/s) using Lyon FTS server
    • Regurlarly cleanup files
  • T2 Role (BEIJING, LAL, LAPP, LPC, LPNHE, SACLAY, TOKYO)
    • Get files from T1 (Lyon). Files on the T2 are written in /home/atlas/sc4tier0/...
    • Clean-up the files (?)
  • Other roles
    • ATLAS (S. Jézéquel, G. Rahal) : Initiate T1->T2 transfers

Information from DDM monitoring



--

Information from FTS monitoring

  • T1->T2 :
    • 15 concurrent files and 10 streams for LYON-TOKYO
    • 5 concurrent files and 5 streams for LYON-BEIJING (SE not enough powerful for 15/15 )
    • 10 concurrent files and 10 streams for LYON-French T2s (LAL, LPNHE, LPC, SACLAY) except for LAPP (5 concurrent files and 1 stream)

Information from dCache monitoring (provided by Lyon)

VOBOX Configuration

  • 4 processors 3 GHz
  • 4 GB of memory ( 2 GB dedicated to SWAP)
  • The monitoring of the Vobox daily, weekly and monthly can be found Here

Disk space availability

Daily news


  • SC4 transfers from LYON to T2s:
    • Simultaneous transfers to the 7 T2 sites for more than 24 hours; it reached more than 25MB/s
    • StackdayLYON-27-7.png

Post-mortem DDM meeting

Page principale du Twiki ATLAS