Monday, February 9, 2009

Aligning VMFS partition

Proper alignment of filesystem on disk partition may bring some I/O performance improvements. Typically, the reason for it is caused by creating RAID device underneath the accessed disk which can stripe data in chunks of some defined size. The typical size of chunk is 64KB. As you know, no partition is placed at the raw beginning of disk because there needs to be written some metadata like MBR or partition table. It is clear now that default aligning may results in latency increase and so in lower throughput.

The same holds for VMFS filesytem, for both versions 2 and 3. The general rule is to align VMFS partition on the 64KB boundary. The problem is default partition alignment by VMware ESX installer (or Red Hat Anaconda). It doesn't count with it and it layouts the disk partitions one by one. If you create VMFS filesystem from VirtualCenter client, it starts from 64KB. Follows output of fdisk -lu command from testing system:
Disk /dev/sda: 146.6 GB, 146685296640 bytes
255 heads, 63 sectors/track, 17833 cylinders, total 286494720 sectors
Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 63 208844 104391 83 Linux
/dev/sda2 208845 10442249 5116702+ 83 Linux
/dev/sda3 10442250 281105369 135331560 fb Unknown
/dev/sda4 281105370 286487144 2690887+ f Win95 Ext'd (LBA)
/dev/sda5 281105433 282213854 554211 82 Linux swap
/dev/sda6 282213918 286294364 2040223+ 83 Linux
/dev/sda7 286294428 286487144 96358+ fc Unknown

Disk /dev/sdb: 128.8 GB, 128849018880 bytes
255 heads, 63 sectors/track, 15665 cylinders, total 251658240 sectors
Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 128 251658224 125829048+ fb Unknown
The first disk /dev/sda is internal one and it was partitioned by ESX installer. The VMFS partition has ID fb. The second disk was initialized from VirtualCenter. It belongs to an external disk array. The starting sector is 128, so it is aligned to 128 x 512B (sector size) = 64KB. The first VMFS partition on /dev/sda is not aligned because 10442250 divided by 128 doesn't give an integer.

There is no destructive way how to realign not optimally aligned VMFS partitions. You need to recreate the partitions from scratch. It requires to backup ESX system, VMFS filesystems, realign the partitions and restore the backup.

There is not defined that every disk or disk array has the alignment boundary at 64KB. It is required to discuss it with system guides. But 64KB is good starting point and it is the most common value. The question is if it is worthwile to perform it because average performance benefit is around 10%.

I drew on more comprehensive guide about the topic published at It contains details about test environment, guest filesystem alignment or steps how to layout partitions with fdisk so read it if you are interested in.

No comments: