Difference between revisions of "Storage-Benches"

Un article de lcgwiki.
Jump to: navigation, search
(DELL Systems)
(Procedures)
 
(6 intermediate revisions by 2 users not shown)
Ligne 1: Ligne 1:
Storage benchmarking and tuning depends upon several parameters like file systems, kernel parameters, RAID configuration, network parameters, etc. This page aims to provide benchmarking tools and procedures for LCG Storage Element, as well as some tuning hints for particular hardware solutions.
+
Storage benchmarking and [[TCP-Tuning|tuning]] depends upon several parameters like file systems, kernel parameters, RAID configuration, network parameters, etc. This page aims to provide benchmarking tools and procedures for LCG Storage Element, as well as some results for particular hardware solutions.
 
=  Benchmarking =
 
  
= Tuning =
+
= Tools =
  
== DELL Systems ==
+
The XIO software developed at CC-IN2P3 have been used to bench the storage. A presentation of this tool has been given in [https://indico.in2p3.fr/event/4340/session/2/contribution/3/material/slides/0.pdf this talk]. Now it is advised to move to [http://git.kernel.dk/cgit/fio/tree/README FIO] with a set of agreed parameters (work in progress).
=== R510 + PowerVault MD1200 ===
 
*RAID avec stripe size de 1m et règle « lecture anticipée adaptative »
 
  
*FS xfs avec alignement des partitions et montage avec l'option "noatime"
+
= Procedures =
  
Exemple:
+
* Describe the hardware
#parted  /dev/sdb mklabel gpt
+
* Indicate filesystem options (at filesystem creation and mount options)
#parted /dev/sdb mkpart primary xfs 1m 50%
+
* Indicate System parameters regarding I/O
#parted /dev/sdb mkpart primary xfs 50% 100%
+
* Execute each test 3 times, report the average and standard deviation
#######
+
* For each test report FIO parameters
mkfs.xfs -d su=1m,sw=10 /dev/sdb1 -L R510_sdb1
+
* Publish your result (format to be defined) with date and person to contact
mkfs.xfs -d su=1m,sw=10 /dev/sdb2 -L R510_sdb2
 
#######
 
# cat /etc/fstab | grep noatime
 
LABEL=R510_sdb1          /fs1                    xfs    defaults,noatime        0 0
 
LABEL=R510_sdb2          /fs2                    xfs    defaults,noatime        0 0
 
  
 +
= Results =
  
*kernel:  
+
* [https://indico.in2p3.fr/event/11617/session/5/contribution/30/material/slides/0.pdf DELL R730xd with 6TB disks] with XIO
**Utilisation du Read ahead system
+
* [https://indico.in2p3.fr/event/11973/session/10/contribution/27/material/slides/0.pdf DELL R730xd with 8TB disks] with XIO
**modif du scheduleur par défaut et du nombre de requete en queue
 
  
# tail -6 /etc/rc.d/rc.local
+
= External links =
blockdev --setra 16384 /dev/sdb
 
blockdev --setra 16384 /dev/sdc
 
echo 512 > /sys/block/sdb/queue/nr_requests
 
echo 512 > /sys/block/sdc/queue/nr_requests
 
echo deadline > /sys/block/sdb/queue/scheduler
 
echo deadline > /sys/block/sdc/queue/scheduler
 
  
== HP Systems ==
+
[http://git.kernel.dk/cgit/fio/tree/README FIO]
 
 
=== MDS 600 based system ===
 
 
 
In order to enhance the performance of the MDS 600 based systems, the following tunings have been applied:
 
* Upgrading the hard disk firmware (HPD3). All hard disks should have the same firmware version.
 
* Changing the power management profile (max cpu power).
 
* Using the following system parameters for each disk:
 
<pre>
 
  echo "cfq" > /sys/block/cciss\!${disk}/queue/scheduler
 
  echo 256 > /sys/block/cciss\!${disk}/queue/nr_requests
 
  echo 4096 >  /sys/block/cciss\!${disk}/queue/read_ahead_kb
 
</pre>
 
* Using XFS filesystem aligned with the RAID strip
 
* Using RAID units composed of 5 disks wisely selected on different columns.
 

Latest revision as of 08:11, 22 septembre 2016

Storage benchmarking and tuning depends upon several parameters like file systems, kernel parameters, RAID configuration, network parameters, etc. This page aims to provide benchmarking tools and procedures for LCG Storage Element, as well as some results for particular hardware solutions.

Tools

The XIO software developed at CC-IN2P3 have been used to bench the storage. A presentation of this tool has been given in this talk. Now it is advised to move to FIO with a set of agreed parameters (work in progress).

Procedures

  • Describe the hardware
  • Indicate filesystem options (at filesystem creation and mount options)
  • Indicate System parameters regarding I/O
  • Execute each test 3 times, report the average and standard deviation
  • For each test report FIO parameters
  • Publish your result (format to be defined) with date and person to contact

Results

External links

FIO