Difference between revisions of "Atlas:FDR2"

Un article de lcgwiki.
Jump to: navigation, search
(8th June)
(8th June)
Ligne 278: Ligne 278:
 
=== 8th June ===
 
=== 8th June ===
 
;14h20
 
;14h20
* FDR2 AODs/DPDs/TAGs/ESDs are being exported to LYON.
+
* FDR2 AODs/DPDs/TAGs/ESDs are being exported to LYON (IN2P3-CC_DATADISK), but haven't been shipped to T2s.
  
 
;Number of datasets transfered by far
 
;Number of datasets transfered by far
Ligne 284: Ligne 284:
 
  68
 
  68
  
;Throughput(snapshot)(MB/s)
+
;Throughput(T0-T1,snapshot)(MB/s)
 
[[Image:LYON.throughput.14400 200806081420.png]]
 
[[Image:LYON.throughput.14400 200806081420.png]]
  
 
;Number of transfer errors
 
;Number of transfer errors
 
[[Image:LYON.num file xs error.14400 200806081420.png]]
 
[[Image:LYON.num file xs error.14400 200806081420.png]]
 +
 +
;Throughput(T1-T2,snapshot)(MB/s)
 +
[[Image:LYON.T2.throughput.14400 T1-T2 200806081420.png]]
  
 
=== 9th June ===
 
=== 9th June ===

Version du 13:41, 8 juin 2008

Monitoring pages/graphs

FTS

http://cctoolsafs.in2p3.fr/fts/monitoring/prod/ftsmonitor.php?vo=atlas

T0-T1 (ALL)

FDR monitoring page - T1s

http://atldq2pro.cern.ch:8000/ft/mon/fdrmon_tier1s.html

http://dashb-atlas-data-tier0.cern.ch/dashboard/request.py/site

  • Throughput
http://dashb-atlas-data-tier0.cern.ch/dashboard/templates/plots/OVERVIEW.throughput.14400.png
  • Errors
http://dashb-atlas-data-tier0.cern.ch/dashboard/templates/plots/OVERVIEW.num_file_xs_error.14400.png

T0-T1 (Lyon)

http://dashb-atlas-data-tier0.cern.ch/dashboard/request.py/site?statsInterval=4&name=LYON

  • Throughput T0-LYON
http://dashb-atlas-data-tier0.cern.ch/dashboard/templates/plots/LYON.throughput.14400.png
  • Errors T0-LYON
http://dashb-atlas-data-tier0.cern.ch/dashboard/templates/plots/LYON.num_file_xs_error.14400.png
  • Throughput Others-LYON
http://dashb-atlas-data.cern.ch/dashboard/templates/plots/LYON.T1.throughput.14400.png

T1-T1

T1-T2 (Lyon)

FDR monitoring page

http://atldq2pro.cern.ch:8000/ft/mon/fdrmon_TiersInfo.html

http://atldq2pro.cern.ch:8000/ft/mon/fdrmon_T1-T2_matrix_day.html

Dashboard

http://dashb-atlas-data.cern.ch/dashboard/request.py/site?statsInterval=4&name=LYON

  • Throughput
http://dashb-atlas-data.cern.ch/dashboard/templates/plots/LYON.T2.throughput.14400.png
  • Errors
http://dashb-atlas-data.cern.ch/dashboard/templates/plots/LYON.T2.num_file_xs_error.14400.png

Network Graphs

IN2P3 Weathermap
cc-in2p3 
lyo-cern (lhcopn-in2p3.cern.ch)
http://netstat.in2p3.fr/weathermap/graphiques/lyo-cern-daily.gif
lyon-nord 
lyo-nrd
http://netstat.in2p3.fr/weathermap/graphiques/lyo-nrd-daily.gif
orsay 
orsay (in2p3-orsay.cssi.renater.fr
http://netstat.in2p3.fr/weathermap/graphiques/orsay-daily.gif
lal-cc 
Liaison LAL - CC
http://netstat.in2p3.fr/weathermap/graphiques/lal-cc-daily.gif
lal 
lal
http://netstat.in2p3.fr/weathermap/graphiques/lal-daily.gif
lpnhe
lpnhe-nrd (in2p3-jussieu.cssi.renater.fr)
http://netstat.in2p3.fr/weathermap/graphiques/lpnhe-nrd-daily.gif
lpnhe (Paris-LPNHE.in2p3.fr)
http://netstat.in2p3.fr/weathermap/graphiques/lpnhe-daily.gif
lapp 
ann-nrd
http://netstat.in2p3.fr/weathermap/graphiques/ann-nrd-daily.gif
lpc
lpc-cf

http://netstat.in2p3.fr/weathermap/graphiques/lpc-cf-daily.gif

lpsc 
lpsc
http://netstat.in2p3.fr/weathermap/graphiques/lpsc-daily.gif
cppm 
cppm

http://netstat.in2p3.fr/weathermap/graphiques/cppm-daily.gif

international (tokyo etc.) 
parisnrd

http://netstat.in2p3.fr/weathermap/graphiques/parisnrd-daily.gif

GEANT - NYC 
MANLAN

http://dc-snmp.wcc.grnoc.iu.edu/manlan/img/sw.newy32aoa.manlan.internet2.edu--te10_1-std5.gif

NYC - TOKYO 
MANLAN

http://dc-snmp.wcc.grnoc.iu.edu/manlan/img/sw.newy32aoa.manlan.internet2.edu--te11_1-std5.gif

tokyo 
gridftp monitor
http://www.icepp.jp/monitor/gridftp/img/all.png

T0-T1

T1-T2

General informations

FDR page

https://twiki.cern.ch/twiki/bin/view/Atlas/FullDressRehearsal

DDM_in_FRD2
https://twiki.cern.ch/twiki/bin/view/Atlas/DDMOperationsGroup#DDM_in_FRD2
Replication status(checked ~ every 2h?)

http://pandamon.usatlas.bnl.gov:25880/server/pandamon/query?mode=listFDR

Lyon FTS monitor
http://cctoolsafs.in2p3.fr/fts/monitoring/prod/ftsmonitor.php?vo=atlas
Contacts
  • IN2P3-CPPM_DATADISK :
  • IN2P3-LPSC_DATADISK :
  • IN2P3-LPC_DATADISK :
  • GRIF-LAL_DATADISK : (grid.admin a lal.in2p3.fr)
  • GRIF-LPNHE_DATADISK :
  • GRIF-SACLAY_DATADISK :
  • BEIJING-LCG2_DATADISK : (yanxf a ihep.ac.cn, Erming.Pei a cern.ch)
  • RO-07-NIPNE_DATADISK : (ciubancan a nipne.ro)
  • RO-02-NIPNE_DATADISK : (tpreda a nipne.ro)
  • TOKYO-LCG2_DATADISK : (lcg-admin a icepp.s.u-tokyo.ac.jp)
Shares

Only srmv2+ space tokens sites will collect data. The list of streams is : Egamma, Bphysics, MinBias, Muon and Jet. If no precise stream name is associated to the share, the site will get any stream. The site in parenthesis will collect the complementary datasets. The T2 ressources at T1 means that users without production role can access CPUs at the T1 without the Production role.

   * US :
         o AGLT2_DATADISK : 100 %
         o MWT2_DATADISK: 100 %
         o NET2_DATADISK: 100 %
         o SLACXRD_DATADISK: 100 %
         o SWT2_CPB_DATADISK: 100 %
         o T2 CPU ressources in BNL to access AOD 
   * CA :
         o SFU-LCG2_DATADISK : 40%
         o VICTORIA-LCG2_DATADISK : 30%
         o ALBERTA-LCG2_DATADISK : 30%
         o TORONTO-LCG2_DATADISK : 50 % 
   * IT :
         o INFN-MILANO_DATADISK : 25%
         o INFN-FRASCATI_DATADISK : 25%
         o INFN-ROMA1_DATADISK : 100%
         o INFN-NAPOLI-ATLAS_DATADISK : 100 % 
   * TW :
         o TW-FTT_DATADISK : 50 % 
   * NL
         o JINR-LCG2_DATADISK : 50% Jet (RU-PROTVINO-IHEP_DATADISK)
         o JINR-LCG2_DATADISK : 50% MinBias (RU-PROTVINO-IHEP_DATADISK)
         o JINR-LCG2_DATADISK : 50% Muon (RU-PROTVINO-IHEP_DATADISK)
         o JINR-LCG2_DATADISK : 50% Bphysics (RU-PNPI_DATADISK)
         o RU-PROTVINO-IHEP_DATADISK : 50% Jet (JINR-LCG2_DATADISK)
         o RU-PROTVINO-IHEP_DATADISK : 50% Muon (JINR-LCG2_DATADISK)
         o RU-PROTVINO-IHEP_DATADISK : 50% MinBias (JINR-LCG2_DATADISK)
         o RU-PROTVINO-IHEP_DATADISK : 50% Egamma (RU-PNPI_DATADISK)
         o RU-PNPI_DATADISK : 50% Egamma (RU-PROTVINO-IHEP_DATADISK)
         o RU-PNPI_DATADISK : 50% Bphysics (JINR-LCG2_DATADISK)
         o RU-PNPI_DATADISK : 50% Jet (RRC-KI_DATADISK)
         o RU-PNPI_DATADISK : 50% MinBias (RRC-KI_DATADISK)
         o RRC-KI_DATADISK : 50% Jet (RU-PNPI_DATADISK)
         o RRC-KI_DATADISK : 50% MinBias (RU-PNPI_DATADISK)
         o RRC-KI_DATADISK : 50% Egamma
         o RRC-KI_DATADISK : 50% Bphysics
         o CSTCDIE_DATADISK : 20% 
   * FR :
         o IN2P3-LAPP_DATADISK : 100 % Egamma
         o IN2P3-CPPM_DATADISK : 100 % Egamma
         o IN2P3-LPSC_DATADISK : 5 %
         o IN2P3-LPC_DATADISK : 100% Muon, 50% Egamma
         o BEIJING-LCG2_DATADISK : 100 % Egamma, 50% Muon, 35% MinBias
         o RO-07-NIPNE_DATADISK : 10%
         o RO-02-NIPNE_DATADISK : 10%
         o GRIF-LAL_DATADISK : 45%
         o GRIF-LPNHE_DATADISK : 25%
         o GRIF-SACLAY_DATADISK : 30 %
         o TOKYO-LCG2_DATADISK : 100%
         o T2 CPU ressources in LYON to access AOD 
   * DE :
         o DESY-HH_DATADISK : 100%
         o CSCS-LCG2_DATADISK: 100%
         o DESY-ZN_DATADISK : 50%
         o GOEGRID_DATADISK : 50%
         o WUPPERTALPROD_DATADISK: 25%
         o CYFRONET-LCG2_DATADISK : 15%
         o PRAGUELCG2_DATADISK : 25%
         o LRZ-LMU_DATADISK : 35%
         o UNI-FREIBURG_DATADISK : 100% Jet, 100% Muon 
   * UK :
         o UKI-SOUTHGRID-CAM-HEP_DATADISK : 50 % Muon (UKI-NORTHGRID-LANCS-HEP_DATADISK
         o UKI-NORTHGRID-LANCS-HEP_DATADISK : 50 % Muon (UKI-SOUTHGRID-CAM-HEP_DATADISK)
         o UKI-NORTHGRID-SHEF-HEP_DATADISK : 50 % MinBias (UKI-SCOTGRID-GLASGOW_DATADISK)
         o UKI-SCOTGRID-GLASGOW_DATADISK : 50 % MinBias (UKI-NORTHGRID-SHEF-HEP_DATADISK)
         o UKI-LT2-RHUL_DATADISK : 50 % Egamma (UKI-NORTHGRID-LIV-HEP_DATADISK)
         o UKI-NORTHGRID-LIV-HEP_DATADISK : 50 % Egamma (UKI-LT2-RHUL_DATADISK)
         o UKI-SOUTHGRID-RALPP_DATADISK : 50 Egamma (UKI-LT2-RHUL_DATADISK)
         o UKI-SOUTHGRID-BHAM-HEP_DATADISK : 50 Jet (UKI-SOUTHGRID-OX-HEP_DATADISK) 
   * ES :
         o IFIC-LCG2_DATADISK: 50%
         o UAM-LCG2_DATADISK: 25%
         o IFAE_DATADISK: 25% 
   * NDGF :
         o SIGNET_DATADISK : 50 %

Logbook

2nd June

  • Since Lyon is in donwtime, so there's no transfers from LYON to T2s today.

http://cc.in2p3.fr/cc_accueil.php3?lang=fr

  • Free Space
Token             Total  Guaranteed Unused
------------------------------------------
BEIJING-LCG2_DATADISK 6.366 6.366 6.343
GRIF-LAL_DATADISK 6.0 6.0 5.732
GRIF-LPNHE_DATADISK 3.906 3.906 3.89
GRIF-SACLAY_DATADISK 6.0 6.0 5.994
IN2P3-CPPM_DATADISK 1.0 1.0 0.977
IN2P3-LAPP_DATADISK 4.0 4.0 4.0
IN2P3-LPC_DATADISK 3.0 3.0 3.0
IN2P3-LPSC_DATADISK 0.488 0.488 0.482
RO-02-NIPNE_DATADISK 3.0 3.0 3.0
RO-07-NIPNE_DATADISK 2.0 2.0 2.0
TOKYO-LCG2_DATADISK 15.0 15.0 13.643

3rd June

  • Lyon is still in donwtime, no transfers from LYON to T2s in this morning.
Throughput(snapshot) (MB/s) 
LYON.throughput.14400 20080603.png
Number of transfer errors(snapshot) 
LYON.num file xs error.14400 20080603.png

4th June

22h02
  • Still no transfers, desipte that all the services in LYON shows OK.
Throughput(snapshot)

LYON.throughput.3600 20080604.png

Services in LYON

Services cc-in2p3 2008060422h15.PNG

Free space
BEIJING-LCG2_DATADISK 6.366 6.366 6.343
BEIJING-LCG2_MCDISK 5.457 5.457 5.457
GRIF-LAL_DATADISK 6.0 6.0 5.732
GRIF-LAL_MCDISK 7.0 7.0 3.724
GRIF-LPNHE_DATADISK 3.906 3.906 3.89
GRIF-LPNHE_MCDISK 1.0 1.0 0.194
GRIF-SACLAY_DATADISK 6.0 6.0 5.994
GRIF-SACLAY_MCDISK 2.0 2.0 1.713
IN2P3-CPPM_DATADISK 1.0 1.0 0.977
IN2P3-CPPM_MCDISK 0.781 0.781 0.765
IN2P3-LAPP_DATADISK 4.0 4.0 4.0
IN2P3-LAPP_MCDISK 2.0 2.0 0.745
IN2P3-LPC_DATADISK 3.0 3.0 3.0
IN2P3-LPC_MCDISK 2.0 2.0 1.936
IN2P3-LPSC_DATADISK 0.488 0.488 0.482
IN2P3-LPSC_MCDISK 2.0 2.0 1.518
RO-02-NIPNE_DATADISK 3.0 3.0 3.0
RO-02-NIPNE_MCDISK 1.0 1.0 0.995
RO-07-NIPNE_DATADISK 2.0 2.0 2.0
RO-07-NIPNE_MCDISK 1.0 1.0 0.654
TOKYO-LCG2_DATADISK 15.0 15.0 13.643
TOKYO-LCG2_MCDISK 10.0 10.0 10.0

5th June

  • Data produced on Tuesday are supposed to be shipped in this night.
14h50
  • Now the RAW data has begun to shipped to LYON.
Throughput(snapshot)(MB/s)

LYON.throughput.14400 RAW 20080605 1816.png

Number of transfer errors (snapshot)

LYON.num file xs error.14400 20080605 1823.png

21h30
  • IN2P3-CC_MCDISK has some problem of staging file. But it should have nothing to do with T0 export.
  • 47 datasets have been exported to T1s. The total is 90.
Throughput(snapshot)(MB/s)

LYON.throughput.14400 20080605 2140.png

Number of transfer errors (snapshot) -- It's the problem of staging file in T1 and nothing to do with FDR transfers.

LYON.num file xs error.14400 20080605 2140.png

6th June

09h50
Throughput(snapshot) (MB/s)

LYON.throughput.14400 200806060 0958.png

Number of transfer errors (snapshot)

LYON.num file xs error.14400 20080606 0959.png

10h03
  • Now all the RAW datasets have been exported to T1. The total number is 90.

http://atlas.web.cern.ch/Atlas/tzero/prod2/monitoring/tables/ddm_RAW_status.html

7th June

  • at CERN, will re-start the bulk processing with using the upcoming AtlasPoint1-14.1.0.12 release cache.
11h30
  • SRM in LYON is still in bad shape. Experts are working on it.

8th June

14h20
  • FDR2 AODs/DPDs/TAGs/ESDs are being exported to LYON (IN2P3-CC_DATADISK), but haven't been shipped to T2s.
Number of datasets transfered by far
dq2-list-dataset-site IN2P3-CC_DATADISK|grep fdr08_run2.*|wc -l
68
Throughput(T0-T1,snapshot)(MB/s)

LYON.throughput.14400 200806081420.png

Number of transfer errors

LYON.num file xs error.14400 200806081420.png

Throughput(T1-T2,snapshot)(MB/s)

LYON.T2.throughput.14400 T1-T2 200806081420.png

9th June

10th June

11st June

12nd June