Alignment problem is well known and causes multiply IO requests for every single OS request. This saturates IO channels ; performance degrades. Alignment of VMFS done by selecting correct volume type on storage level. Lets suppose that all done well and VMDK file beguns exactly on storage block start. Modern disks (especially SSD) and storages works with 4k block size (pages) emulation 512 block access. This emulating costs in performance degradation. We'll try to bring things back to 4k.
Starting point: you have Linux installed on single vmdk file, like:
# fdisk -l /dev/sda Disk /dev/sda: 8589 MB, 8589934592 bytes 255 heads, 63 sectors/track, 1044 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 13 104391 83 Linux /dev/sda2 14 1044 8281507+ 8e Linux LVMwhere /boot (and GRUB) sits in /dev/sda1 and MBR, and rest of OS hidden by LVM in /dev/sda2. I am going to separate these partitions to different disks, achieving easy resizing as side effect.
Add (on-line) two new disks (vmdk) to Linux VM. One small disk to fit /boot only (150m enough) and second to hold rest OS (mine fits in 8g). I'd recommend to keep application data on separate disk (vmdk file). This makes management, clones, templates much easy. First disk would be not align, but it works at boot time only, so impact minimal.
Rescan devices in Linux:
# for SH in /sys/class/scsi_host/host?/scan ; do echo "- - -" > $SH ; done # fdisk -l ... Disk /dev/sdb: 157 MB, 157286400 bytes 64 heads, 32 sectors/track, 150 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Disk /dev/sdb doesn't contain a valid partition table Disk /dev/sdc: 8589 MB, 8589934592 bytes 255 heads, 63 sectors/track, 1044 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/sdc doesn't contain a valid partition table
Lets transfer LVM to aligned volume. Resulting disk is unpartitioned with data aligned to 4k boundary.
# pvcreate --dataalignment 4k /dev/sdc Physical volume "/dev/sdc" successfully created # pvs PV VG Fmt Attr PSize PFree /dev/sda2 rootvg lvm2 a- 7.88G 2.78G /dev/sdc lvm2 a- 8.00G 8.00G # vgextend rootvg /dev/sdc Volume group "rootvg" successfully extended # pvmove /dev/sda2 /dev/sda2: Moved: 0.0% ... # pvs PV VG Fmt Attr PSize PFree /dev/sda2 rootvg lvm2 a- 7.88G 7.88G /dev/sdc rootvg lvm2 a- 7.97G 2.88G # vgreduce rootvg /dev/sda2 Removed "/dev/sda2" from volume group "rootvg" # pvremove /dev/sda2 Labels on physical volume "/dev/sda2" successfully wiped
Transfer /boot and GRUB:
# # DANGEROUS !! THIS WILL REBUILD PARTITION TABLE on /dev/sdb !!! # echo -e ",,,,\n;\n;\n;" | sfdisk /dev/sdb # mkfs.ext3 -j -m0 -L /boot /dev/sdb1 # mkdir /boot.new ; mount /dev/sdb1 /boot.new # (cd /boot && tar cf - .)|(cd /boot.new && tar xpvfB -) # grub grub> device (hd1) /dev/sdb grub> root (hd1,0) Filesystem type is ext2fs, partition type 0x83 grub> setup (hd1) Checking if "/boot/grub/stage1" exists... no Checking if "/grub/stage1" exists... yes Checking if "/grub/stage2" exists... yes Checking if "/grub/e2fs_stage1_5" exists... yes Running "embed /grub/e2fs_stage1_5 (hd1)"... failed (this is not fatal) Running "embed /grub/e2fs_stage1_5 (hd1,0)"... failed (this is not fatal) Running "install /grub/stage1 (hd1) /grub/stage2 p /grub/grub.conf "... succeeded Done. grub> quit
It is time to turn off VM, remove old disk and change SCSI ids for new disks, set 0 for small disk and 1 for bigger one.
Add (on-line) disk for data (mine 30g). Set it's SCSI id to be (1:0). This will create also "New SCSI Controller" Change it's type to VMware Paravirtual. This type is not so sutable for OS disks, because it limits upgrading possibility, however it good for data/application.
# for SH in /sys/class/scsi_host/host?/scan ; do echo "- - -" > $SH ; done # fdisk -l ... Disk /dev/sdc: 32.2 GB, 32212254720 bytes 64 heads, 32 sectors/track, 30720 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Disk /dev/sdc doesn't contain a valid partition table
Create aligned PV, VG, and format FS with 4k blosk size:
# pvcreate --dataalignment 4k /dev/sdc Physical volume "/dev/sdc" successfully created # vgcreate orahome /dev/sdc Volume group "orahome" successfully created # lvcreate -n export -L25g /dev/orahome Logical volume "export" created # mkfs.ext3 -j -m0 -b4096 /dev/orahome/export # mkdir /export && mount /dev/orahome/export /export
Paravirtual SCSI controller starts after FSCK checks, then its FS should be marked as _netdev at /etc/fstab file:
# grep orahome /etc/fstab /dev/orahome/export /export ext3 _netdev 2 2
Due to the fact, that LVM does NOT resides on partition now, there is no need in partition resizing. Then reboot could be avoided and system disk extend can be performed on-line. Change OS disk size to 10g now. Rescan:
# for device in /sys/block/sd* ; do echo 1 > $device/device/rescan ; sleep 2 ; done # pvs PV VG Fmt Attr PSize PFree /dev/sdb rootvg lvm2 a- 7.97G 2.88G /dev/sdc orahome lvm2 a- 30.00G 5.00G # pvresize /dev/sdb Physical volume "/dev/sdb" changed 1 physical volume(s) resized / 0 physical volume(s) not resized # pvs PV VG Fmt Attr PSize PFree /dev/sdb rootvg lvm2 a- 9.97G 4.88G /dev/sdc orahome lvm2 a- 30.00G 5.00G
The rest is trivial.