Atlas:Functionnal-test-lyon-oct06

Un article de lcgwiki.
Jump to navigation Jump to search

The goal of this test is to check that data produced can be moved between sites (functionnalities are working). It is not a performance measurement yet.

News and actions

  • 27 October:
    • Problem of BNL-LYON resolved (pending since 18 October)
    • All FTS submissions are blocked in 'Submit' mode. Requested intervention by LYON team
  • 26 October :
    • D. Cameron corrected DDM monitoring because files already transfered during T0->T1 are not monitored when doing T2->T1. But induced a bug in the speed transfer (80 MB/s LYON->T2 is wrong). Bug is corrected

but not yet validated.

  • 25 October :
    • It looks that T2->T1 transfers went well. But no measurement because of limitation in DDM monitoring
    • Transfer of 3 GB files from CERN to T1s. OK for LYON but problems in dcache for files > 2 GB appeared

(ls -l says 1 B) and dccp does not work in BNL.

  • 24 October :
    • FTS server in Lyon seems to be overloaded (appear 'File exists' in T2)
    • One of the dcache servers for LYONDISK (but were are the data of this test) cannot sustain the load ('gridftp errors')
    • Results for the FR cloud:

LYON - 396 files transferred 

       Beijing, LAL done
       CPPM  50% done (50% ASSIGNED)
       LAPP  70% done (30% in progress)
       LPC   30%      (70%)
       LPNHE 20%      (80%)
       SACLAY60%      (40%)
       TOKYO 70%      (30%)
  • 23 October :
    • BNL accessible again (routing problem in Lyon)
    • All ddmf1 files reached LYONDISK ( ~40 files per dataset)
  • 22 October : Start of the test
  • 21 October :
    • No possible transfer between T2s and Lyon T1 (all FTS jobs in Pending mode)
    • No contact with BNL from LYON (FTS, srm-get-metadata, afs). Not the case for TRIUMF or if test is done from CERN
  • 19 October : Available disk space:
    • sedpm.mrs.grid.cnrs.fr : 188 GB
    • lpnse1.in2p3.fr : 2790 GB
    • lapp-se01.in2p3.fr 804 GB
    • clrlcgse03.in2p3.fr: 2270 GB
    • grid05.lal.in2p3.fr: 1159 GB
    • node12.datagrid.cea.fr 3760 GB
    • dgse0.icepp.jp: 6760 GB
    • atlasse01.ihep.ac.cn: 1018 GB

Ressource monitoring

Summary

mail from Alexei Klimentov (7 October 2006)

  • The next DDM functional test is scheduled for October 2006 after SC4
  • We will start to generate data in a week from now (Oct 15th)
  • Please update table 'Sites' (DDM ops WiKi Functional test) for your site. Data will be generated only for the sites listed in the table. (DDM ops WiKi : https://twiki.cern.ch/twiki/bin/view/Atlas/DDMOperations)
  • Test duration 7 - 10 days
  • The test scope (preliminary) :
repeat Step1 and Step2 of the previous test
Step 1: CERN->Tier1->Tier2s data transfer
Step 2: Tier2s->Tier1 data transfer

Bulk data transfer

CERN -> Tier1s
Tier1->Tier2s (2-3 Tier-2s per cloud)
  • Data transfer of large 2+GB files (if available)
  • Evaluation of data transfer robustness vs dataset size
  • FTS open access (data transfer between different clouds)
  • Two more functional tests are already scheduled Year 2006 (week 47-48, TBC), and Jan 2007. Probably in January we will try to evaluate data transfer performance.


  • several suggestions, comments, questions
    • will we use DDM ops mailing list to discuss test scope or HyperNews ?
    • Tier-1s will need 200-300GB disk space for the October test
    • Tier-2s (if they will participate in bulk data transfer) will need 100-200 GB disk space for the October test
    • There are no special requirements for the SW installation. Run what you are using for SC4

Page principale du Twiki ATLAS

Récupérée de « https://lcg.in2p3.fr/index.php?title=Atlas:Functionnal-test-lyon-oct06&oldid=2432 »