Creating a large storage volume on a Linux VM running on an ESXi host

I have an ESXi host in my office with a RAiD10 array that has around 925GB in capacity – 95% unused. I have a couple of VMs on it that hardly use any of that space and I wanted to make use of all the spare storage space to make a backup of our family photos which were previously stored on two 500GB external drives and take up over 700GB. That’s quite a lot isn’t it? … you might say. Well, I agree, but my wife is rather trigger happy and loves to take 10 slightly different versions of the same shot so that she can choose the best one…. unfortunately the choosing of the best ones rarely involves deleting more than 1 out of each 10 of what was taken, hence the monstrously large library. Ah well!

The ideal scenario was to create a single storage space that could hold the entire library without having to split it into smaller parts if possible. However, the ESXi host has a limit of 256GB per virtual hard disk due to the disk block size of the datastore being set to 1MB.

After doing some research I realised that it should be possible to add extra virtual hard disks to my Centos VM and then use them to create an LVM volume group that could be mounted as a single partition and used to store the library.

Here are the steps I took:

  1. Added three new hard disks – each 256GB in size to the Centos VM in the VSphere client. This tutorial was helpful: http://www.matttopper.com/?p=25
  2. 3newHDs

  3. Restarted the VM so that Centos picks up the new virtual disks
  4. dev

  5. Formatted each of the disks like so:

    # fdisk /dev/sdb

    • ‘n’ – new partition
    • ‘p’ – primary partition
    • ’1′ – first partition on disk
    • ‘enter’ – First cylinder (default 1)
    • ‘enter’ – Last cylinder (default 33418)
    • ‘t’ – select partition 1 and set the ID
    • ’8e’ – change system type of partition to Linux LVM
    • ‘w’ – save changes and exit

    I repeated this for each of the three disks.

  6. Created a ‘Physical Volume’ from the three virtual disks

    [root@CENTOS-OPENBD ~]# pvcreate /dev/sdb1 /dev/sdc1 /dev/sdd1
    Physical volume "/dev/sdb1" successfully created
    Physical volume "/dev/sdc1" successfully created
    Physical volume "/dev/sdd1" successfully created
  7. Checked the PVs

    [root@CENTOS-OPENBD ~]# pvscan
    PV /dev/sda2 VG VolGroup00 lvm2 [7.88 GB / 0 free]
    PV /dev/sdb1 lvm2 [255.99 GB]
    PV /dev/sdc1 lvm2 [255.99 GB]
    PV /dev/sdd1 lvm2 [255.99 GB]
    Total: 4 [775.86 GB] / in use: 1 [7.88 GB] / in no VG: 3 [767.98 GB]
  8. Initialised the /etc/lvmtab and /etc/lvmtab.d files
    [root@CENTOS-OPENBD ~]# vgscan
    Reading all physical volumes. This may take a while...
    Found volume group "VolGroup00" using metadata type lvm2
  9. Create the volume group and add the new PVs. I used a 16Mb block size here so that the volume group could scale up to 1TB (the default setting would mean a limit of 256 which is the reason why I’m having to do this in the first place – the ESXi datasource uses 1MB blocks)
    [root@CENTOS-OPENBD ~]# vgcreate -s 16M centosOpenBD_vol_grp /dev/sdb1 /dev/sdc1 /dev/sdd1
    Volume group "centosOpenBD_vol_grp" successfully created
  10. Create a logical volume in the new volume group
    [root@CENTOS-OPENBD ~]# lvcreate -L 760G -n Foto_Library centosOpenBD_vol_grp
    Logical volume "Foto_Library" created
  11. Check the results
    [root@CENTOS-OPENBD ~]# lvdisplay
    --- Logical volume ---
    LV Name /dev/centosOpenBD_vol_grp/Foto_Library
    VG Name centosOpenBD_vol_grp
    LV UUID dGGGl9-8sLh-CPGX-LeCt-aANt-2sHw-Dr1y2e
    LV Write Access read/write
    LV Status available
    # open 0
    LV Size 760.00 GB
    Current LE 48640
    Segments 3
    Allocation inherit
    Read ahead sectors auto
    - currently set to 256
    Block device 253:2
    --- Logical volume ---
    LV Name /dev/VolGroup00/LogVol00
    VG Name VolGroup00
    LV UUID i0rWpk-jru6-xjs6-0V1d-1yOD-I3jf-8Q2xwS
    LV Write Access read/write
    LV Status available
    # open 1
    LV Size 3.97 GB
    Current LE 127
    Segments 1
    Allocation inherit
    Read ahead sectors auto
    - currently set to 256
    Block device 253:0
    --- Logical volume ---
    LV Name /dev/VolGroup00/LogVol01
    VG Name VolGroup00
    LV UUID UKAKAT-ZOK5-xmsi-27an-G9QO-RvOB-nPGbBF
    LV Write Access read/write
    LV Status available
    # open 1
    LV Size 3.91 GB
    Current LE 125
    Segments 1
    Allocation inherit
    Read ahead sectors auto
    - currently set to 256
    Block device 253:1
  12. Make the filesystem on the new logical volume (this step takes quite a long time to complete – about ten minutes in my case)
    [root@CENTOS-OPENBD ~]# mke2fs -j /dev/centosOpenBD_vol_grp/Foto_Library
    mke2fs 1.39 (29-May-2006)
    Filesystem label=
    OS type: Linux
    Block size=4096 (log=2)
    Fragment size=4096 (log=2)
    99614720 inodes, 199229440 blocks
    9961472 blocks (5.00%) reserved for the super user
    First data block=0
    Maximum filesystem blocks=0
    6080 block groups
    32768 blocks per group, 32768 fragments per group
    16384 inodes per group
    Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
    102400000
    Writing inode tables: done
    Creating journal (32768 blocks): done
    Writing superblocks and filesystem accounting information: done
    This filesystem will be automatically checked every 21 mounts or
    180 days, whichever comes first. Use tune2fs -c or -i to override.
  13. Mount the filesystem
    mount /dev/centosOpenBD_vol_grp/Foto_Library /data
  14. Add a line to /etc/fstab:
    /dev/centosOpenBD_vol_grp/Foto_Library /data ext3 defaults 0 0

Now I have a nice big volume permanently mounted in my server that can be used to store lots of photos.

More reading:
http://sequires.com/blog/?p=17
http://sujithemmanuel.blogspot.com/2007/04/how-to-add-disk-to-lvm.html

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>