Filesystem Block Sizes On Hardware RAID

Interesting note from IBM on setting up filesystem blocksizes (or software RAID block sizes) on hardware RAID:

If you plan to use RAID devices in your file system, a larger block size may be more effective and help avoid the penalties involved in small block write operations to RAID devices. For example, in a RAID configuration using 4 data disks and 1 parity disk (a 4+P configuration), which uses a 64 KB stripe size, the optimal file system block size would be an integral multiple of 256 KB (4 data disks × 64 KB stripe size = 256 KB). A block size of an integral multiple of 256 KB results in a single data write that encompasses the 4 data disks and a parity-write to the parity disk. If a block size smaller than 256 KB, such as 64 KB, is used, write performance is degraded by the read-modify-write behavior. A 64 KB block size results in a single disk writing 64 KB and a subsequent read from the three remaining disks in order to compute the parity that is then written to the parity disk. The extra read degrades performance.

The choice of block size also affects the performance of certain metadata operations, in particular, block allocation performance. The GPFS block allocation map is stored in blocks, similar to regular files. When the block size is small: – It takes more blocks to store a given amount of data resulting in additional work to allocate those blocks – One block of allocation map data contains less information.

 Basically, when nesting storage blocks, use higher multiples of each inner layer's block size for performance gains.  Welcome to storage 101.