Difference between revisions of "Atlas:StorageRequirements"

Un article de lcgwiki.
Jump to: navigation, search
Ligne 66: Ligne 66:
 
| '''<code>.../atlas/<locality>/user</code>''' <br> write permission to local ATLAS users (<code>/atlas/<locality></code>)
 
| '''<code>.../atlas/<locality>/user</code>''' <br> write permission to local ATLAS users (<code>/atlas/<locality></code>)
 
|}
 
|}
 +
 +
 +
* A T3 will need ATLASDATADISK if it would like to receive real data,  ATLASMCDISK to received simulated data, ATLASGROUPDISK to receive group analysis data, and ATLASPRODDISK to contribute official production.  ATLASUSERDISK is not necessary, but may be needed if the site contributes non-local user analysis.
 +
 +
  
 
== ATLASDATADISK ==
 
== ATLASDATADISK ==
Ligne 86: Ligne 91:
 
default:other::r-x
 
default:other::r-x
 
</pre>
 
</pre>
 
* A T3 will need ATLASDATADISK if it would like to receive real data,  ATLASMCDISK to received simulated data, ATLASGROUPDISK to receive group analysis data, and ATLASPRODDISK to contribute official production.  ATLASUSERDISK is not necessary, but may be needed if the site contributes non-local user analysis.
 
 
  
 
== ATLASMCDISK ==
 
== ATLASMCDISK ==

Version du 17:11, 7 juillet 2008

Introduction

  • The requirements for ATLASGROUPDISK is temporary due to a technical limitation in DPM. Atlas has contacted DPM developpers. The proper implementation is expected to be available in September.


  • Unless mentioned, the size is for a typical T2 with ~500 CPUs and ~100 TB disk
  • Site admin's are requested to create at least the space tokens. The namespace directories can be left to atlas production if the top directory is properly set up so that /atlas/Role=production is allowed to execute dpns-mkdir and dpns-setacl remotely. (Of course they are welcome to create them by themselves).


  • In general read permission should be given to all ATLAS users everywhere.
  • In general write permission should be given to /atlas/Role=production everywhere, so that the ATLAS central deletion tool can work.
  • It is suggested at LAPP to give write permission to atlas/Role=lcgadmin everywhere, so that the ATLAS admin can control the storage. (less people in Role=lcgadmin than in than in /atlas/Role=production). Stephane will discuss this issue with ATLAS.


ATLAS top directory

  • On this page the top directory for ATLAS is represented by .../atlas. Each site should replace it according to their SE configuration.
    • eg. lapp-se01.in2p3.fr:/dpm/in2p3.fr/home/atlas for lapp.
  • Although it is up to sites policies, but it is recommended
    • to have this top directory configured so that only /atlas/Role=production and /atlas/Role=lcgadmin can create files/directories underneath and ordinary users cannot.


Summary table for Space token and Name space

space token voms group to be associated space to be reserved namespace directory(-ies) to be created
ATLASDATADISK /atlas/Role=production see below* .../atlas/atlasdatadisk

write permission only to /atlas/Role=production

ATLASMCDISK /atlas/Role=production 15TB
(60TB for 100% AOD) see below*
.../atlas/atlasmcdisk
write permission only to atlas/Role=production
ATLASPRODDISK /atlas/Role=production 2TB (see below*) .../atlas/atlasproddisk
write permission only to /atlas/Role=production
ATLASGROUPDISK /atlas (temporary solution. see below*) 6TB (see below*) .../atlas/atlasgroupdisk
write permission to /atlas/Role=production


.../atlas/atlasgroupdisk/$GROUP
write permission to /atlas/Role=production and /atlas/$GROUP/Role=production

($GROUP: see below*)

ATLASUSERDISK /atlas 5TB (see below*) .../atlas/user
write permission to all ATLAS users
write permission to /atlas/Role=production for central deletion
ATLASLOCALGROUPDISK /atlas/<locality>, <locality>=fr,ro,cn,jp,... sites to decide .../atlas/<locality>/user
write permission to local ATLAS users (/atlas/<locality>)


  • A T3 will need ATLASDATADISK if it would like to receive real data, ATLASMCDISK to received simulated data, ATLASGROUPDISK to receive group analysis data, and ATLASPRODDISK to contribute official production. ATLASUSERDISK is not necessary, but may be needed if the site contributes non-local user analysis.


ATLASDATADISK

  • Estimation of size comes later
  • VOMS group associated with the space: /atlas/Role=production
  • Namespace directory to be created: .../atlas/atlasdatadisk
    • Normally, sites have already this namespace created.
  • Namespace ACL: writable by only atlas/Role=production, readable by all ATLAS users
# group: atlas/Role=production
user::rwx
group::rwx              #effective:rwx
group:atlas/Role=production:rwx         #effective:rwx
mask::rwx
other::r-x
default:user::rwx
default:group::rwx
default:group:atlas/Role=production:rwx
default:mask::rwx
default:other::r-x

ATLASMCDISK

  • 15TB for a typical T2 with ~500 CPU?fs and ~100 TB disk
    • 60TB for a T2 requesting for 100% AOD.
    • The size may increase in case D1PD comes in this space.
  • VOMS group associated with the space: /atlas/Role=production
  • Namespace directory to be created: /atlas/atlasmcdisk
    • Normally, sites have already this namespace created.
  • Namespace ACL: writable by only atlas/Role=production, readable by all ATLAS users
    # group: atlas/Role=production
    user::rwx
    group::rwx              #effective:r-x
    group:atlas/Role=production:rwx         #effective:rwx
    mask::rwx
    other::r-x
    default:user::rwx
    default:group::rwx
    default:group:atlas/Role=production:rwx
    default:mask::rwx
    default:other::r-x
    

ATLASPRODDISK

  • 2TB for a typical T2 with ~500 CPU?fs (the size is to be re-visited)
    • scales with the CPU capacity of the site.
    • will be larger if the reconstruction jobs run on the site.
  • VOMS group associated with the space: /atlas/Role=production
  • Namespace directory to be created: .../atlas/atlasproddisk
  • Namespace ACL: writable by only atlas/Role=production, readable by all ATLAS users
# group: atlas/Role=production
user::rwx
group::rwx              #effective:rwx
group:atlas/Role=production:rwx         #effective:rwx
mask::rwx
other::r-x
default:user::rwx
default:group::rwx
default:group:atlas/Role=production:rwx
default:mask::rwx
default:other::r-x

ATLASGROUPDISK

  • 6TB for a typical T2 with ~500 CPU?fs and ~100 TB disk
    • The size may decrease in case D1PD goes into ATLASMCDISK.
  • One single space to be reserved for all the group activities ($GROUP = phys-beauty, phys-exotics, phys-gener, phys-hi, phys-higgs, phys-lumin, phys-sm, phys-susy, phys-top, perf-egamma, perf-flavtag, perf-jets, perf-muons, perf-tau, etc.)
  • VOMS group associated with the space: /atlas
    • A temporary solution until multiple group support to the spaces is available.
    • Once it is available, the groups will be /atlas/Role=production and /atlas/$GROUP/Role=production for all $GROUP
  • Namespaces directory to be created and their ACLs:
    • .../atlas/atlasgroupdisk: writable by only atlas/Role=production, readable by all ATLAS users
    • .../atlas/atlasgroupdisk/$GROUP: writable by atlas/Role=production and /atlas/$GROUP/Role=production, readable by all ATLAS users
    • eg. for phys-higgs
    # group: atlas/Role=production
    user::rwx
    group::rwx              #effective:rwx
    group:atlas/Role=production:rwx         #effective:rwx
    group:atlas/phys-higgs/Role=production:rwx         #effective:rwx
    mask::rwx
    other::r-x
    default:user::rwx
    default:group::rwx
    default:group:atlas/Role=production:rwx
    default:group:atlas/phys-higgs/Role=production:rwx
    default:mask::rwx
    default:other::r-x
    </ul>


    ATLASUSERDISK

    • 5TB for a typical T2 with ~500 CPU?fs and ~100 TB disk
    • VOMS group associated with the space: /atlas
    • Namespace directory to be created: .../atlas/user
    • Namespace ACL: writable by all ATLAS users
    • Normally, sites have already this namespace created by user analysis jobs so far.
    • example commands
      • dpns-mkdir .../atlas/user
      • dpns-setacl -m g:atlas:rwx,m:rwx,d:g:atlas:r-x .../atlas/user
      • dpns-getacl .../atlas/user
      # group: atlas
      user::rwx
      group::rwx              #effective:rwx
      group:atlas/Role=production:rwx         #effective:rwx
      mask::rwx
      other::r-x
      default:user::rwx
      default:group::r-x
      default:group:atlas/Role=production:rwx
      default:mask::rwx
      default:other::r-x
      


    ATLASLOCALGROUPDISK

    • size to be decided by sites.
      • the resources not included in the pledge.
    • VOMS group associated with the space: /atlas/fr (or /atlas/ro, /atlas/cn, /atlas/jp correspondingly)
    • name space: .../atlas/fr/user (or .../atlas/ro/user, .../atlas/cn/user, .../atlas/jp/user, etc.)
    • ACL: write permission only to /atlas/fr group (or /atlas/ro, /atlas/cn, /atlas/jp correspondingly)
    • example ACL:
      % DPNS_HOST=lapp-se01.in2p3.fr; dpns-getacl $DPNS_HOST:/dpm/in2p3.fr/home/atlas/atlaslocalgroupdisk
      # file: lapp-se01.in2p3.fr:/dpm/in2p3.fr/home/atlas/atlaslocalgroupdisk
      # owner: root
      # group: atlas/Role=production
      user::rwx
      group::r-x              #effective:r-x
      group:atlas/Role=lcgadmin:rwx           #effective:rwx
      group:atlas/fr:rwx              #effective:rwx
      mask::rwx
      other::r-x
      default:user::rwx
      default:group::rwx
      default:group:atlas/Role=lcgadmin:rwx
      default:group:atlas/fr:rwx
      default:mask::rwx
      default:other::r-x