Atlas

Un article de lcgwiki.
Revision as of 17:41, 2 décembre 2018 by Jezequel (talk | contribs) (→‎DOMA_FR tests)
Jump to: navigation, search

Bienvenue sur la page Atlas LCG-France
Welcome to the LCG-France Atlas page

General DOMA informations

DOMA_FR project

DOMA_FR tests

  • Global transfers from/ to (same site dest/source site excluded)
Header text LAPP LPSC CC
From

Link

Link

Link

To

Link

Link

Link

  • Data transfer through LHCONE
Transfer Sud-East CC
Source

https://monit-grafana.cern.ch/d/FtSFfwdmk/ddm-transfers?orgId=17&from=now-7d&to=now&panelId=32&fullscreen&var-activity=All&var-src_cloud=All&var-src_site=IN2P3-CPPM&var-src_site=IN2P3-LAPP&var-src_site=IN2P3-LPC&var-src_site=IN2P3-LPSC&var-src_tier=All&var-src_endpoint=All&var-dst_cloud=All&var-dst_site=AGLT2&var-dst_site=ANLASC&var-dst_site=AYDIN&var-dst_site=Australia-ATLAS&var-dst_site=BEIJING-LCG2&var-dst_site=BNL-ATLAS&var-dst_site=BOINC&var-dst_site=BU_ATLAS_Tier2&var-dst_site=CA-SFU-T2&var-dst_site=CA-VICTORIA-WESTGRID-T2&var-dst_site=CA-WATERLOO-T2&var-dst_site=CERN-PROD&var-dst_site=CSCS-LCG2&var-dst_site=CYFRONET-LCG2&var-dst_site=DESY-HH&var-dst_site=DESY-ZN&var-dst_site=EELA-UTFSM&var-dst_site=FMPhI-UNIBA&var-dst_site=FZK-LCG2&var-dst_site=GOOGLE&var-dst_site=GoeGrid&var-dst_site=HEPHY-UIBK&var-dst_site=HK-LCG2&var-dst_site=IEPSAS-Kosice&var-dst_site=IFIC-LCG2&var-dst_site=IL-TAU-HEP&var-dst_site=INFN-BOLOGNA-T3&var-dst_site=INFN-COSENZA&var-dst_site=INFN-FRASCATI&var-dst_site=INFN-GENOVA&var-dst_site=INFN-LECCE&var-dst_site=INFN-MILANO-ATLASC&var-dst_site=INFN-NAPOLI-ATLAS&var-dst_site=INFN-ROMA1&var-dst_site=INFN-ROMA2&var-dst_site=INFN-ROMA3&var-dst_site=INFN-T1&var-dst_site=INFN-TRIESTE&var-dst_site=ITEP&var-dst_site=JINR-LCG2&var-dst_site=LRZ-LMU&var-dst_site=LUCILLE&var-dst_site=MPPMU&var-dst_site=MWT2&var-dst_site=NCG-INGRID-PT&var-dst_site=NDGF-T1&var-dst_site=NERSC-PDSF&var-dst_site=NIKHEF-ELPROD&var-dst_site=Nevis&var-dst_site=OU_OSCER_ATLAS&var-dst_site=PSNC&var-dst_site=RAL-LCG2&var-dst_site=RO-02-NIPNE&var-dst_site=RO-07-NIPNE&var-dst_site=RO-14-ITIM&var-dst_site=RO-16-UAIC&var-dst_site=RRC-KI&var-dst_site=RRC-KI-T1&var-dst_site=RU-Protvino-IHEP&var-dst_site=SARA-MATRIX&var-dst_site=SE-SNIC-T2&var-dst_site=SMU_HPC&var-dst_site=SWT2_CPB&var-dst_site=SiGNET&var-dst_site=TECHNION-HEP&var-dst_site=TOKYO-LCG2&var-dst_site=TR-10-ULAKBIM&var-dst_site=TRIUMF-LCG2&var-dst_site=TW-FTT&var-dst_site=Taiwan-LCG2&var-dst_site=UAM-LCG2&var-dst_site=UKI-LT2-Brunel&var-dst_site=UKI-LT2-IC-HEP&var-dst_site=UKI-LT2-QMUL&var-dst_site=UKI-LT2-RHUL&var-dst_site=UKI-NORTHGRID-LANCS-HEP&var-dst_site=UKI-NORTHGRID-LIV-HEP&var-dst_site=UKI-NORTHGRID-MAN-HEP&var-dst_site=UKI-NORTHGRID-SHEF-HEP&var-dst_site=UKI-SCOTGRID-DURHAM&var-dst_site=UKI-SCOTGRID-ECDF&var-dst_site=UKI-SCOTGRID-GLASGOW&var-dst_site=UKI-SOUTHGRID-BHAM-HEP&var-dst_site=UKI-SOUTHGRID-CAM-HEP&var-dst_site=UKI-SOUTHGRID-OX-HEP&var-dst_site=UKI-SOUTHGRID-RALPP&var-dst_site=UKI-SOUTHGRID-SUSX&var-dst_site=UNI-BONN&var-dst_site=UNI-FREIBURG&var-dst_site=UNI-SIEGEN-HEP&var-dst_site=UNIBE-LHEP&var-dst_site=USTC-T3&var-dst_site=UTA_SWT2&var-dst_site=WEIZMANN-LCG2&var-dst_site=WT2&var-dst_site=ZA-CHPC&var-dst_site=ZA-WITS-CORE&var-dst_site=ifae&var-dst_site=mainz&var-dst_site=mainzgrid&var-dst_site=pic&var-dst_site=praguelcg2&var-dst_site=ru-PNPI&var-dst_site=wuppertalprod&var-dst_tier=All&var-dst_endpoint=All&var-binning=$__auto_interval_binning&var-groupby=activity&var-columns=dst_cloud&var-rows=src_cloud%7C&var-measurement=ddm_transfer&var-retention_policy=raw || https://monit-grafana.cern.ch/d/FtSFfwdmk/ddm-transfers?orgId=17&from=now-7d&to=now&panelId=32&fullscreen&var-activity=All&var-src_cloud=All&var-src_site=IN2P3-CC&var-src_tier=All&var-src_endpoint=All&var-dst_cloud=All&var-dst_site=AGLT2&var-dst_site=ANLASC&var-dst_site=AYDIN&var-dst_site=Australia-ATLAS&var-dst_site=BEIJING-LCG2&var-dst_site=BNL-ATLAS&var-dst_site=BOINC&var-dst_site=BU_ATLAS_Tier2&var-dst_site=CA-SFU-T2&var-dst_site=CA-VICTORIA-WESTGRID-T2&var-dst_site=CA-WATERLOO-T2&var-dst_site=CERN-PROD&var-dst_site=CSCS-LCG2&var-dst_site=CYFRONET-LCG2&var-dst_site=DESY-HH&var-dst_site=DESY-ZN&var-dst_site=EELA-UTFSM&var-dst_site=FMPhI-UNIBA&var-dst_site=FZK-LCG2&var-dst_site=GOOGLE&var-dst_site=GoeGrid&var-dst_site=HEPHY-UIBK&var-dst_site=HK-LCG2&var-dst_site=IEPSAS-Kosice&var-dst_site=IFIC-LCG2&var-dst_site=IL-TAU-HEP&var-dst_site=INFN-BOLOGNA-T3&var-dst_site=INFN-COSENZA&var-dst_site=INFN-FRASCATI&var-dst_site=INFN-GENOVA&var-dst_site=INFN-LECCE&var-dst_site=INFN-MILANO-ATLASC&var-dst_site=INFN-NAPOLI-ATLAS&var-dst_site=INFN-ROMA1&var-dst_site=INFN-ROMA2&var-dst_site=INFN-ROMA3&var-dst_site=INFN-T1&var-dst_site=INFN-TRIESTE&var-dst_site=ITEP&var-dst_site=JINR-LCG2&var-dst_site=LRZ-LMU&var-dst_site=LUCILLE&var-dst_site=MPPMU&var-dst_site=MWT2&var-dst_site=NCG-INGRID-PT&var-dst_site=NDGF-T1&var-dst_site=NERSC-PDSF&var-dst_site=NIKHEF-ELPROD&var-dst_site=Nevis&var-dst_site=OU_OSCER_ATLAS&var-dst_site=PSNC&var-dst_site=RAL-LCG2&var-dst_site=RO-02-NIPNE&var-dst_site=RO-07-NIPNE&var-dst_site=RO-14-ITIM&var-dst_site=RO-16-UAIC&var-dst_site=RRC-KI&var-dst_site=RRC-KI-T1&var-dst_site=RU-Protvino-IHEP&var-dst_site=SARA-MATRIX&var-dst_site=SE-SNIC-T2&var-dst_site=SMU_HPC&var-dst_site=SWT2_CPB&var-dst_site=SiGNET&var-dst_site=TECHNION-HEP&var-dst_site=TOKYO-LCG2&var-dst_site=TR-10-ULAKBIM&var-dst_site=TRIUMF-LCG2&var-dst_site=TW-FTT&var-dst_site=Taiwan-LCG2&var-dst_site=UAM-LCG2&var-dst_site=UKI-LT2-Brunel&var-dst_site=UKI-LT2-IC-HEP&var-dst_site=UKI-LT2-QMUL&var-dst_site=UKI-LT2-RHUL&var-dst_site=UKI-NORTHGRID-LANCS-HEP&var-dst_site=UKI-NORTHGRID-LIV-HEP&var-dst_site=UKI-NORTHGRID-MAN-HEP&var-dst_site=UKI-NORTHGRID-SHEF-HEP&var-dst_site=UKI-SCOTGRID-DURHAM&var-dst_site=UKI-SCOTGRID-ECDF&var-dst_site=UKI-SCOTGRID-GLASGOW&var-dst_site=UKI-SOUTHGRID-BHAM-HEP&var-dst_site=UKI-SOUTHGRID-CAM-HEP&var-dst_site=UKI-SOUTHGRID-OX-HEP&var-dst_site=UKI-SOUTHGRID-RALPP&var-dst_site=UKI-SOUTHGRID-SUSX&var-dst_site=UNI-BONN&var-dst_site=UNI-FREIBURG&var-dst_site=UNI-SIEGEN-HEP&var-dst_site=UNIBE-LHEP&var-dst_site=USTC-T3&var-dst_site=UTA_SWT2&var-dst_site=WEIZMANN-LCG2&var-dst_site=WT2&var-dst_site=ZA-CHPC&var-dst_site=ZA-WITS-CORE&var-dst_site=ifae&var-dst_site=mainz&var-dst_site=mainzgrid&var-dst_site=pic&var-dst_site=praguelcg2&var-dst_site=ru-PNPI&var-dst_site=wuppertalprod&var-dst_tier=2&var-dst_tier=3&var-dst_endpoint=All&var-binning=$__auto_interval_binning&var-groupby=activity&var-columns=dst_cloud&var-rows=src_cloud%7C&var-measurement=ddm_transfer&var-retention_policy=raw

Destination
  • Computing activity per job type (Running slots)
Running slots LAPP LPSC CC

[1]

[2]

[3]

  • Data access monitorings as seen by site WN
Destination of access LAPP LPSC CC
Production download

Link

Link

Link

Production upload

Link

Link

Link

Production input

Link

Link

Link

Production output

Link

Link

Link

Analysis Download

Link

Link

Link

Analysis Direct Access

link

Link

Link

  • Questions
    • Enabling direct access creates much more network usage -> Is it usefull for ATLAS ? (processing urgent request faster ?)
    • Production_output can only be done to 1 site
    • If direct access to IN2P3-CC in direct access with xrootd, what is the size of the portal (2 or 10 Gb/s for ATLAS)


  • Issues
    • Asynchronous transfers of input files goes to closest/fastest site to input site instead of the read_lan0 to the WN (would help to reduce network occupancy between IN2P3-CC and LAPP/LPSC since smoothed by FTS) -> Request to change on 21st September (Panda level)
    • Jobs brokering should take into account downtime of remote SEs (issue with IN2P3-CC downtime) -> Request sent by Rod
    • 10 Gb/s connection LAPP-CC (used for all LAPP WAN transfers to any site) can get saturated if huge amount of jobs starting at same time (No smoothing by Panda) -> No suggestion yet
    • Production_output can only be done to 1 site (usefull if SE destination in downtime while local/remote WN is not) -> No request
    • Users can enforce analysis jobs to copy files instead of direct access -> the whole file is transfered instead of a fraction
    • IN2P3-CC is running only 200-300 analysis jobs while the total site runs 10k : Reason is the different analysis share between T1 (5% ) and T2 (25% ?)
    • IN2P3-CC presents remote direct read access because read_wan0=srm


  • Next steps
    • Deploy Rucio 1.17 to use protocol priority defined in AGIS (bug in 1.16). Solving bug : use most trusted protocol and would help to control srm decreasing usage
    • Monitor job efficiency vs RTT between SE and WN -> Identify when cache does not have impact (assuming no network bandwidth limitation)
    • Understand the ATLAS job brokering algorithm
    • Get the typical transfer rate per job type (Johannes)
Récupérée de « http://lcg.in2p3.fr/wiki/index.php?title=Atlas&oldid=8450 »