Difference between revisions of "Atlas"

Un article de lcgwiki.
Jump to: navigation, search
(DOMA_FR tests)
Ligne 28: Ligne 28:
 
  ||  
 
  ||  
 
[https://monit-grafana.cern.ch/d/NppG4Rhiz/atlas-ddm-dashboard-latest-data?orgId=17&from=now-7d&to=now&var-groupby=activity&var-binning=$__auto_interval_binning&var-activity=All&var-src_country=All&var-src_site=AGLT2&var-src_site=ANLASC&var-src_site=Australia-ATLAS&var-src_site=BEIJING-LCG2&var-src_site=BNL-ATLAS&var-src_site=BOINC&var-src_site=BU_ATLAS_Tier2&var-src_site=CA-SFU-T2&var-src_site=CA-VICTORIA-WESTGRID-T2&var-src_site=CA-WATERLOO-T2&var-src_site=CERN-PROD&var-src_site=CSCS-LCG2&var-src_site=CYFRONET-LCG2&var-src_site=DESY-HH&var-src_site=DESY-ZN&var-src_site=EELA-UTFSM&var-src_site=FMPhI-UNIBA&var-src_site=FZK-LCG2&var-src_site=GOOGLE&var-src_site=GRIF&var-src_site=GoeGrid&var-src_site=HEPHY-UIBK&var-src_site=HK-LCG2&var-src_site=IEPSAS-Kosice&var-src_site=IFIC-LCG2&var-src_site=IL-TAU-HEP&var-src_site=IN2P3-LAPP&var-src_site=IN2P3-CPPM&var-src_site=IN2P3-LPC&var-src_site=IN2P3-LPSC&var-src_site=INFN-BOLOGNA-T3&var-src_site=INFN-COSENZA&var-src_site=INFN-FRASCATI&var-src_site=INFN-GENOVA&var-src_site=INFN-LECCE&var-src_site=INFN-MILANO-ATLASC&var-src_site=INFN-NAPOLI-ATLAS&var-src_site=INFN-ROMA1&var-src_site=INFN-ROMA2&var-src_site=INFN-ROMA3&var-src_site=INFN-T1&var-src_site=INFN-TRIESTE&var-src_site=ITEP&var-src_site=JINR-LCG2&var-src_site=LRZ-LMU&var-src_site=LUCILLE&var-src_site=MPPMU&var-src_site=MWT2&var-src_site=NCG-INGRID-PT&var-src_site=NDGF-T1&var-src_site=NERSC-PDSF&var-src_site=NIKHEF-ELPROD&var-src_site=Nevis&var-src_site=OU_OSCER_ATLAS&var-src_site=PSNC&var-src_site=RAL-LCG2&var-src_site=RO-02-NIPNE&var-src_site=RO-07-NIPNE&var-src_site=RO-14-ITIM&var-src_site=RO-16-UAIC&var-src_site=RRC-KI&var-src_site=RRC-KI-T1&var-src_site=RU-Protvino-IHEP&var-src_site=SARA-MATRIX&var-src_site=SE-SNIC-T2&var-src_site=SWT2_CPB&var-src_site=SiGNET&var-src_site=TECHNION-HEP&var-src_site=TOKYO-LCG2&var-src_site=TR-10-ULAKBIM&var-src_site=TRIUMF-LCG2&var-src_site=TUDresden-ZIH&var-src_site=TW-FTT&var-src_site=Taiwan-LCG2&var-src_site=UAM-LCG2&var-src_site=UKI-LT2-Brunel&var-src_site=UKI-LT2-IC-HEP&var-src_site=UKI-LT2-QMUL&var-src_site=UKI-LT2-RHUL&var-src_site=UKI-NORTHGRID-LANCS-HEP&var-src_site=UKI-NORTHGRID-LIV-HEP&var-src_site=UKI-NORTHGRID-MAN-HEP&var-src_site=UKI-NORTHGRID-SHEF-HEP&var-src_site=UKI-SCOTGRID-DURHAM&var-src_site=UKI-SCOTGRID-ECDF&var-src_site=UKI-SCOTGRID-GLASGOW&var-src_site=UKI-SOUTHGRID-BHAM-HEP&var-src_site=UKI-SOUTHGRID-CAM-HEP&var-src_site=UKI-SOUTHGRID-OX-HEP&var-src_site=UKI-SOUTHGRID-RALPP&var-src_site=UKI-SOUTHGRID-SUSX&var-src_site=UNI-BONN&var-src_site=UNI-FREIBURG&var-src_site=UNI-SIEGEN-HEP&var-src_site=UNIBE-LHEP&var-src_site=UNKNOWN&var-src_site=UTA_SWT2&var-src_site=WEIZMANN-LCG2&var-src_site=WT2&var-src_site=ZA-CHPC&var-src_site=ZA-WITS-CORE&var-src_site=ifae&var-src_site=mainzgrid&var-src_site=pic&var-src_site=praguelcg2&var-src_site=ru-PNPI&var-src_site=wuppertalprod&var-dst_country=All&var-dst_site=IN2P3-CC&var-dst_site=LAPP&var-measurement=ddm_transfer&var-retention_policy=raw&var-columns=dst_cloud&var-rows=src_cloud| Link]
 
[https://monit-grafana.cern.ch/d/NppG4Rhiz/atlas-ddm-dashboard-latest-data?orgId=17&from=now-7d&to=now&var-groupby=activity&var-binning=$__auto_interval_binning&var-activity=All&var-src_country=All&var-src_site=AGLT2&var-src_site=ANLASC&var-src_site=Australia-ATLAS&var-src_site=BEIJING-LCG2&var-src_site=BNL-ATLAS&var-src_site=BOINC&var-src_site=BU_ATLAS_Tier2&var-src_site=CA-SFU-T2&var-src_site=CA-VICTORIA-WESTGRID-T2&var-src_site=CA-WATERLOO-T2&var-src_site=CERN-PROD&var-src_site=CSCS-LCG2&var-src_site=CYFRONET-LCG2&var-src_site=DESY-HH&var-src_site=DESY-ZN&var-src_site=EELA-UTFSM&var-src_site=FMPhI-UNIBA&var-src_site=FZK-LCG2&var-src_site=GOOGLE&var-src_site=GRIF&var-src_site=GoeGrid&var-src_site=HEPHY-UIBK&var-src_site=HK-LCG2&var-src_site=IEPSAS-Kosice&var-src_site=IFIC-LCG2&var-src_site=IL-TAU-HEP&var-src_site=IN2P3-LAPP&var-src_site=IN2P3-CPPM&var-src_site=IN2P3-LPC&var-src_site=IN2P3-LPSC&var-src_site=INFN-BOLOGNA-T3&var-src_site=INFN-COSENZA&var-src_site=INFN-FRASCATI&var-src_site=INFN-GENOVA&var-src_site=INFN-LECCE&var-src_site=INFN-MILANO-ATLASC&var-src_site=INFN-NAPOLI-ATLAS&var-src_site=INFN-ROMA1&var-src_site=INFN-ROMA2&var-src_site=INFN-ROMA3&var-src_site=INFN-T1&var-src_site=INFN-TRIESTE&var-src_site=ITEP&var-src_site=JINR-LCG2&var-src_site=LRZ-LMU&var-src_site=LUCILLE&var-src_site=MPPMU&var-src_site=MWT2&var-src_site=NCG-INGRID-PT&var-src_site=NDGF-T1&var-src_site=NERSC-PDSF&var-src_site=NIKHEF-ELPROD&var-src_site=Nevis&var-src_site=OU_OSCER_ATLAS&var-src_site=PSNC&var-src_site=RAL-LCG2&var-src_site=RO-02-NIPNE&var-src_site=RO-07-NIPNE&var-src_site=RO-14-ITIM&var-src_site=RO-16-UAIC&var-src_site=RRC-KI&var-src_site=RRC-KI-T1&var-src_site=RU-Protvino-IHEP&var-src_site=SARA-MATRIX&var-src_site=SE-SNIC-T2&var-src_site=SWT2_CPB&var-src_site=SiGNET&var-src_site=TECHNION-HEP&var-src_site=TOKYO-LCG2&var-src_site=TR-10-ULAKBIM&var-src_site=TRIUMF-LCG2&var-src_site=TUDresden-ZIH&var-src_site=TW-FTT&var-src_site=Taiwan-LCG2&var-src_site=UAM-LCG2&var-src_site=UKI-LT2-Brunel&var-src_site=UKI-LT2-IC-HEP&var-src_site=UKI-LT2-QMUL&var-src_site=UKI-LT2-RHUL&var-src_site=UKI-NORTHGRID-LANCS-HEP&var-src_site=UKI-NORTHGRID-LIV-HEP&var-src_site=UKI-NORTHGRID-MAN-HEP&var-src_site=UKI-NORTHGRID-SHEF-HEP&var-src_site=UKI-SCOTGRID-DURHAM&var-src_site=UKI-SCOTGRID-ECDF&var-src_site=UKI-SCOTGRID-GLASGOW&var-src_site=UKI-SOUTHGRID-BHAM-HEP&var-src_site=UKI-SOUTHGRID-CAM-HEP&var-src_site=UKI-SOUTHGRID-OX-HEP&var-src_site=UKI-SOUTHGRID-RALPP&var-src_site=UKI-SOUTHGRID-SUSX&var-src_site=UNI-BONN&var-src_site=UNI-FREIBURG&var-src_site=UNI-SIEGEN-HEP&var-src_site=UNIBE-LHEP&var-src_site=UNKNOWN&var-src_site=UTA_SWT2&var-src_site=WEIZMANN-LCG2&var-src_site=WT2&var-src_site=ZA-CHPC&var-src_site=ZA-WITS-CORE&var-src_site=ifae&var-src_site=mainzgrid&var-src_site=pic&var-src_site=praguelcg2&var-src_site=ru-PNPI&var-src_site=wuppertalprod&var-dst_country=All&var-dst_site=IN2P3-CC&var-dst_site=LAPP&var-measurement=ddm_transfer&var-retention_policy=raw&var-columns=dst_cloud&var-rows=src_cloud| Link]
 +
|}
 +
 +
* Computing activity per job type (Running slots)
 +
 +
{| class="wikitable"
 +
|-
 +
! Running slots !! LAPP !! LPSC !! CC
 +
|-
 +
|  ||
 +
[http://dashb-atlas-job.cern.ch/dashboard/request.py/jobnumbers_individual?sites=IN2P3-LAPP&sitesCat=All%20Countries&resourcetype=All&sitesSort=2&sitesCatSort=0&start=null&end=null&timeRange=lastWeek&granularity=Hourly&generic=0&sortBy=12&series=All&type=rmulticores| Link]
 +
||
 +
[http://dashb-atlas-job.cern.ch/dashboard/request.py/jobnumbers_individual?sites=IN2P3-LPSC&sitesCat=All%20Countries&resourcetype=All&sitesSort=2&sitesCatSort=0&start=null&end=null&timeRange=lastWeek&granularity=Hourly&generic=0&sortBy=12&series=All&type=rmulticores| Link]
 +
||
 +
[http://dashb-atlas-job.cern.ch/dashboard/request.py/jobnumbers_individual?sites=IN2P3-CC&sitesCat=All%20Countries&resourcetype=All&sitesSort=2&sitesCatSort=0&start=null&end=null&timeRange=lastWeek&granularity=Hourly&generic=0&sortBy=12&series=All&type=rmulticores| Link]
 
|}
 
|}
  

Version du 10:59, 22 septembre 2018

Bienvenue sur la page Atlas LCG-France
Welcome to the LCG-France Atlas page

DOMA_FR project

DOMA_FR tests

  • Global transfers from/ to (same site dest/source site excluded)
Header text LAPP LPSC CC
From

Link

Link

Link

To

Link

Link

Link

  • Computing activity per job type (Running slots)
Running slots LAPP LPSC CC

Link

Link

Link

  • Data access monitorings as seen by site WN
Destination of access LAPP LPSC CC
Production download

Link

Link

Link

Production upload

Link

Link

Link

Production input

Link

Link

Link

Production output

Link

Link

Link

Analysis Download

Link

Link

Link

Analysis Direct Access

link

Link

Link

  • Questions
    • Enabling direct access creates much more network usage -> Is it usefull for ATLAS ? (processing urgent request faster ?)
    • Production_output can only be done to 1 site


  • Issues
    • Asynchronous transfers of input files goes to closest/fastest site to input site instead of the read_lan0 to the WN (would help to reduce network occupancy between IN2P3-CC and LAPP/LPSC since smoothed by FTS) -> Request to change on 21st September (Panda level)
    • Jobs brokering should take into account downtime of remote SEs (issue with IN2P3-CC downtime) -> Request sent by Rod
    • 10 Gb/s connection LAPP-CC (used for all LAPP WAN transfers to any site) can get saturated if huge amount of jobs starting at same time (No smoothing by Panda) -> No suggestion yet
    • Production_output can only be done to 1 site (usefull if SE destination in downtime while local/remote WN is not) -> No request
    • Users can enforce analysis jobs to copy files instead of direct access -> the whole file is transfered instead of a fraction


  • Next steps
    • Deploy Rucio 1.17 to use protocol priority defined in AGIS (bug in 1.16). Solving bug : use most trusted protocol and would help to control srm decreasing usage
    • Monitor job efficiency vs RTT between SE and WN -> Identify when cache does not have impact (assuming no network bandwidth limitation)
    • Understand the ATLAS job brokering algorithm
    • Get the typical transfer rate per job type (Johannes)