Atlas

Bienvenue sur la page Atlas LCG-France
Welcome to the LCG-France Atlas page

General DOMA informations

• Indico agenda

• Twiki

DOMA_FR project

• Atrium link
• Document for funding request IN2P3 2019-2021 (Restricted access)
• Roadmap (Restricted access)

DOMA_FR tests

• Data Access (read/transfer/write) Nomenclature

• RTT between main sites

• Global transfers from/ to (same site dest/source site excluded)

Header text	LAPP	LPSC	CC
From	Link	Link	Link
To	Link	Link	Link

• Data transfer through LHCONE

Transfer	Sud-East	CC
Source	https://monit-grafana.cern.ch/d/FtSFfwdmk/ddm-transfers?orgId=17&from=now-7d&to=now&panelId=32&fullscreen&var-activity=All&var-src_cloud=All&var-src_site=IN2P3-CPPM&var-src_site=IN2P3-LAPP&var-src_site=IN2P3-LPC&var-src_site=IN2P3-LPSC&var-src_tier=All&var-src_endpoint=All&var-dst_cloud=All&var-dst_site=AGLT2&var-dst_site=ANLASC&var-dst_site=AYDIN&var-dst_site=Australia-ATLAS&var-dst_site=BEIJING-LCG2&var-dst_site=BNL-ATLAS&var-dst_site=BOINC&var-dst_site=BU_ATLAS_Tier2&var-dst_site=CA-SFU-T2&var-dst_site=CA-VICTORIA-WESTGRID-T2&var-dst_site=CA-WATERLOO-T2&var-dst_site=CERN-PROD&var-dst_site=CSCS-LCG2&var-dst_site=CYFRONET-LCG2&var-dst_site=DESY-HH&var-dst_site=DESY-ZN&var-dst_site=EELA-UTFSM&var-dst_site=FMPhI-UNIBA&var-dst_site=FZK-LCG2&var-dst_site=GOOGLE&var-dst_site=GoeGrid&var-dst_site=HEPHY-UIBK&var-dst_site=HK-LCG2&var-dst_site=IEPSAS-Kosice&var-dst_site=IFIC-LCG2&var-dst_site=IL-TAU-HEP&var-dst_site=INFN-BOLOGNA-T3&var-dst_site=INFN-COSENZA&var-dst_site=INFN-FRASCATI&var-dst_site=INFN-GENOVA&var-dst_site=INFN-LECCE&var-dst_site=INFN-MILANO-ATLASC&var-dst_site=INFN-NAPOLI-ATLAS&var-dst_site=INFN-ROMA1&var-dst_site=INFN-ROMA2&var-dst_site=INFN-ROMA3&var-dst_site=INFN-T1&var-dst_site=INFN-TRIESTE&var-dst_site=ITEP&var-dst_site=JINR-LCG2&var-dst_site=LRZ-LMU&var-dst_site=LUCILLE&var-dst_site=MPPMU&var-dst_site=MWT2&var-dst_site=NCG-INGRID-PT&var-dst_site=NDGF-T1&var-dst_site=NERSC-PDSF&var-dst_site=NIKHEF-ELPROD&var-dst_site=Nevis&var-dst_site=OU_OSCER_ATLAS&var-dst_site=PSNC&var-dst_site=RAL-LCG2&var-dst_site=RO-02-NIPNE&var-dst_site=RO-07-NIPNE&var-dst_site=RO-14-ITIM&var-dst_site=RO-16-UAIC&var-dst_site=RRC-KI&var-dst_site=RRC-KI-T1&var-dst_site=RU-Protvino-IHEP&var-dst_site=SARA-MATRIX&var-dst_site=SE-SNIC-T2&var-dst_site=SMU_HPC&var-dst_site=SWT2_CPB&var-dst_site=SiGNET&var-dst_site=TECHNION-HEP&var-dst_site=TOKYO-LCG2&var-dst_site=TR-10-ULAKBIM&var-dst_site=TRIUMF-LCG2&var-dst_site=TW-FTT&var-dst_site=Taiwan-LCG2&var-dst_site=UAM-LCG2&var-dst_site=UKI-LT2-Brunel&var-dst_site=UKI-LT2-IC-HEP&var-dst_site=UKI-LT2-QMUL&var-dst_site=UKI-LT2-RHUL&var-dst_site=UKI-NORTHGRID-LANCS-HEP&var-dst_site=UKI-NORTHGRID-LIV-HEP&var-dst_site=UKI-NORTHGRID-MAN-HEP&var-dst_site=UKI-NORTHGRID-SHEF-HEP&var-dst_site=UKI-SCOTGRID-DURHAM&var-dst_site=UKI-SCOTGRID-ECDF&var-dst_site=UKI-SCOTGRID-GLASGOW&var-dst_site=UKI-SOUTHGRID-BHAM-HEP&var-dst_site=UKI-SOUTHGRID-CAM-HEP&var-dst_site=UKI-SOUTHGRID-OX-HEP&var-dst_site=UKI-SOUTHGRID-RALPP&var-dst_site=UKI-SOUTHGRID-SUSX&var-dst_site=UNI-BONN&var-dst_site=UNI-FREIBURG&var-dst_site=UNI-SIEGEN-HEP&var-dst_site=UNIBE-LHEP&var-dst_site=USTC-T3&var-dst_site=UTA_SWT2&var-dst_site=WEIZMANN-LCG2&var-dst_site=WT2&var-dst_site=ZA-CHPC&var-dst_site=ZA-WITS-CORE&var-dst_site=ifae&var-dst_site=mainz&var-dst_site=mainzgrid&var-dst_site=pic&var-dst_site=praguelcg2&var-dst_site=ru-PNPI&var-dst_site=wuppertalprod&var-dst_tier=All&var-dst_endpoint=All&var-binning=$__auto_interval_binning&var-groupby=activity&var-columns=dst_cloud&var-rows=src_cloud%7C&var-measurement=ddm_transfer&var-retention_policy=raw || https://monit-grafana.cern.ch/d/FtSFfwdmk/ddm-transfers?orgId=17&from=now-7d&to=now&panelId=32&fullscreen&var-activity=All&var-src_cloud=All&var-src_site=IN2P3-CC&var-src_tier=All&var-src_endpoint=All&var-dst_cloud=All&var-dst_site=AGLT2&var-dst_site=ANLASC&var-dst_site=AYDIN&var-dst_site=Australia-ATLAS&var-dst_site=BEIJING-LCG2&var-dst_site=BNL-ATLAS&var-dst_site=BOINC&var-dst_site=BU_ATLAS_Tier2&var-dst_site=CA-SFU-T2&var-dst_site=CA-VICTORIA-WESTGRID-T2&var-dst_site=CA-WATERLOO-T2&var-dst_site=CERN-PROD&var-dst_site=CSCS-LCG2&var-dst_site=CYFRONET-LCG2&var-dst_site=DESY-HH&var-dst_site=DESY-ZN&var-dst_site=EELA-UTFSM&var-dst_site=FMPhI-UNIBA&var-dst_site=FZK-LCG2&var-dst_site=GOOGLE&var-dst_site=GoeGrid&var-dst_site=HEPHY-UIBK&var-dst_site=HK-LCG2&var-dst_site=IEPSAS-Kosice&var-dst_site=IFIC-LCG2&var-dst_site=IL-TAU-HEP&var-dst_site=INFN-BOLOGNA-T3&var-dst_site=INFN-COSENZA&var-dst_site=INFN-FRASCATI&var-dst_site=INFN-GENOVA&var-dst_site=INFN-LECCE&var-dst_site=INFN-MILANO-ATLASC&var-dst_site=INFN-NAPOLI-ATLAS&var-dst_site=INFN-ROMA1&var-dst_site=INFN-ROMA2&var-dst_site=INFN-ROMA3&var-dst_site=INFN-T1&var-dst_site=INFN-TRIESTE&var-dst_site=ITEP&var-dst_site=JINR-LCG2&var-dst_site=LRZ-LMU&var-dst_site=LUCILLE&var-dst_site=MPPMU&var-dst_site=MWT2&var-dst_site=NCG-INGRID-PT&var-dst_site=NDGF-T1&var-dst_site=NERSC-PDSF&var-dst_site=NIKHEF-ELPROD&var-dst_site=Nevis&var-dst_site=OU_OSCER_ATLAS&var-dst_site=PSNC&var-dst_site=RAL-LCG2&var-dst_site=RO-02-NIPNE&var-dst_site=RO-07-NIPNE&var-dst_site=RO-14-ITIM&var-dst_site=RO-16-UAIC&var-dst_site=RRC-KI&var-dst_site=RRC-KI-T1&var-dst_site=RU-Protvino-IHEP&var-dst_site=SARA-MATRIX&var-dst_site=SE-SNIC-T2&var-dst_site=SMU_HPC&var-dst_site=SWT2_CPB&var-dst_site=SiGNET&var-dst_site=TECHNION-HEP&var-dst_site=TOKYO-LCG2&var-dst_site=TR-10-ULAKBIM&var-dst_site=TRIUMF-LCG2&var-dst_site=TW-FTT&var-dst_site=Taiwan-LCG2&var-dst_site=UAM-LCG2&var-dst_site=UKI-LT2-Brunel&var-dst_site=UKI-LT2-IC-HEP&var-dst_site=UKI-LT2-QMUL&var-dst_site=UKI-LT2-RHUL&var-dst_site=UKI-NORTHGRID-LANCS-HEP&var-dst_site=UKI-NORTHGRID-LIV-HEP&var-dst_site=UKI-NORTHGRID-MAN-HEP&var-dst_site=UKI-NORTHGRID-SHEF-HEP&var-dst_site=UKI-SCOTGRID-DURHAM&var-dst_site=UKI-SCOTGRID-ECDF&var-dst_site=UKI-SCOTGRID-GLASGOW&var-dst_site=UKI-SOUTHGRID-BHAM-HEP&var-dst_site=UKI-SOUTHGRID-CAM-HEP&var-dst_site=UKI-SOUTHGRID-OX-HEP&var-dst_site=UKI-SOUTHGRID-RALPP&var-dst_site=UKI-SOUTHGRID-SUSX&var-dst_site=UNI-BONN&var-dst_site=UNI-FREIBURG&var-dst_site=UNI-SIEGEN-HEP&var-dst_site=UNIBE-LHEP&var-dst_site=USTC-T3&var-dst_site=UTA_SWT2&var-dst_site=WEIZMANN-LCG2&var-dst_site=WT2&var-dst_site=ZA-CHPC&var-dst_site=ZA-WITS-CORE&var-dst_site=ifae&var-dst_site=mainz&var-dst_site=mainzgrid&var-dst_site=pic&var-dst_site=praguelcg2&var-dst_site=ru-PNPI&var-dst_site=wuppertalprod&var-dst_tier=2&var-dst_tier=3&var-dst_endpoint=All&var-binning=$__auto_interval_binning&var-groupby=activity&var-columns=dst_cloud&var-rows=src_cloud%7C&var-measurement=ddm_transfer&var-retention_policy=raw	Destination

• Computing activity per job type (Running slots)

Running slots	LAPP	LPSC	CC
	[1]	[2]	[3]

• Data access monitorings as seen by site WN

Destination of access	LAPP	LPSC	CC
Production download	Link	Link	Link
Production upload	Link	Link	Link
Production input	Link	Link	Link
Production output	Link	Link	Link
Analysis Download	Link	Link	Link
Analysis Direct Access	link	Link	Link

• Logbook and access configuration

• Questions
  • Enabling direct access creates much more network usage -> Is it usefull for ATLAS ? (processing urgent request faster ?)
  • Production_output can only be done to 1 site
  • If direct access to IN2P3-CC in direct access with xrootd, what is the size of the portal (2 or 10 Gb/s for ATLAS)

• Issues
  • Asynchronous transfers of input files goes to closest/fastest site to input site instead of the read_lan0 to the WN (would help to reduce network occupancy between IN2P3-CC and LAPP/LPSC since smoothed by FTS) -> Request to change on 21st September (Panda level)
  • Jobs brokering should take into account downtime of remote SEs (issue with IN2P3-CC downtime) -> Request sent by Rod
  • 10 Gb/s connection LAPP-CC (used for all LAPP WAN transfers to any site) can get saturated if huge amount of jobs starting at same time (No smoothing by Panda) -> No suggestion yet
  • Production_output can only be done to 1 site (usefull if SE destination in downtime while local/remote WN is not) -> No request
  • Users can enforce analysis jobs to copy files instead of direct access -> the whole file is transfered instead of a fraction
  • IN2P3-CC is running only 200-300 analysis jobs while the total site runs 10k : Reason is the different analysis share between T1 (5% ) and T2 (25% ?)
  • IN2P3-CC presents remote direct read access because read_wan0=srm

• Next steps
  • Deploy Rucio 1.17 to use protocol priority defined in AGIS (bug in 1.16). Solving bug : use most trusted protocol and would help to control srm decreasing usage
  • Monitor job efficiency vs RTT between SE and WN -> Identify when cache does not have impact (assuming no network bandwidth limitation)
  • Understand the ATLAS job brokering algorithm
  • Get the typical transfer rate per job type (Johannes)