Picture the following scenario: an organization uses EC2 instances for its database where data is stored in EBS volumes. As the company grows, the size of its database also grows at a rate of 10GB per day. In terms of storage, the organization has a 1TB EBS volume, which they can increase to 16TB. But, every time they increase it, a maintenance window has to be scheduled where the database server is shut down, an EBS snapshot of the data volume is created, and a newer, bigger volume is created and attached to the instance.
However, this causes a problem for the organization because shutting down the database server results in lost business. On top of that, there are network performance bottlenecks between EC2 instances and the EBS volume that impact system performance on the whole.
To fix these issues, the organization decides to leverage Logical Volume Managers (LVMs), which provide the option to easily increase the size of their volume by adding more EBS volumes. With multiple EBS volumes, network performance is increased between EC2 instances and EBS volumes.
In this post, we’ll take a deeper look at dynamic disk arrays (i.e., LVM).
LVMs create an abstraction layer on top of underlying disk layouts and allows sysadmins to create virtual partitions over physical volumes. You can have multiple virtual partitions on a physical volume or a single virtual partition over multiple physical volumes. LVMs provide the ability to dynamically expand and shrink virtual partitions and make additions to physical volumes on the fly for existing LVMs.
How to Configure an LVM Volume
- Launch an EC2 instance
- Attach EBS volumes to your EC2 instanceIn this case, we are attaching three 10GB SSD EBS volumes. The EBS volumes are given the following device names: ‘/dev/sdf’, ‘/dev/sdg’ and ‘/dev/sdh’.
- Initialize devices to use with the LVMIn order to do this, we need to use the pvcreate command. pvcreate is used to initialize disk or partitions that will be used by LVM. It can either initialize a complete physical disk or a partition on physical disks.Syntax : sudo pvcreate <device-1> <device-2> <device-3>
- Display device attributesFor this, we need to use the pvdisplay command to display information about physical disks.
- Create the LVM volume group. Once the devices are initialized, you can create the LVM volume group. A group of physical volumes or disks are combined together into a single storage file which is referred to as the LVM volume group. For this, we used the vgcreate command to create a volume group by providing the name of the volume group and the path of actual physical volumes.Syntax: # sudo vgcreate <volume-name> <device-1> <device-2> <device-3>
Once the LVM volume group is created, you can use the vgdisplay command to show its attributes.
There are also commands like “vgscan”, which scan all disks for LVM volume groups.
- Create Logical VolumesOnce the LVM volume group is created, it’s time to create logical volumes.Syntax: sudo lvcreate –name <logical-volume-name> –size <size-of-volume> <lvm-volume-name>In this case, we are creating two logical volumes (data and backup).For data logical volume:
For backup logical volume:
Once the logical volumes are created, we can view their status using the “lvdisplay” command.
- Format your logical volumes. Once the logical volumes are created, we can format them using any filesystem like ext4, XFS, and more. If you are using the ext4 filesystem:#mkfs.ext4 <logical-volume-path>In our case, it’s:# mkfs.ext4 /dev/n2ws/data
# mkfs.ext4 /dev/n2ws/backup
- Mount the logical volumes. Once the logical volumes are formatted, you can mount them on your mount directories. Please make sure your mount directories already exist. If not, please create them.
Use the mount command to mount:# mount <logical-volume> <mount-point>
If you want you mount points to be available after reboot, you can add mount point entries in ‘/etc/fstab’.
To view the status of the mount, use the mount command.
Once the logical volumes are created and mounted, you can use them as your normal volumes. However, LVM offers:
- Better performance – If your data is spread across multiple EBS volumes using LVM, you can leverage dedicated network throughput between EC2 instances wand EBS volumes. This provides you better network throughput over a single network channel between EC2 and EBS volumes.
- The ability to grow – You can expand your volume at any time according to your requirement. More EBS volumes can be added to your LVM volume rather than creating a snapshot of an EBS volume and expanding it.
- EBS volume snapshots – You need to ensure that there are no operations happening in your EBS volume during snapshots. You can suspend operations on your LVM volume by using the dmsetup command.
Syntax for suspend : # dmsetup suspend <lvm-volume-name>
Syntax for resume : # dmsetup resume <lvm-volume-name>
As mentioned above, LVM volume backup can be done using the EBS snapshot procedure, ensuring that LVM volume operations are suspended for that duration of time. To understand more about LVM backup concepts, check out “Disk Array Backup on EC2 Part I: Concepts“. If you are interested in understanding consistency challenges with LVM backups, take a look at “Disk Array Backup on EC2 Part II: Consistency Issues“.
In order to perform LVM EBS snapshot backups, see “Disk Array Backup on EC2 Part III: LVM backup with EBS Volumes“.
N2Ws offers a solution that enables you to further simplify the process of using snapshots.
N2WS Backup & Recovery (CPM) is an enterprise-class backup-recovery and disaster recovery solution for the EC2 compute cloud. CPM is available as a service model that allows users to manage multiple AWS accounts and configure policies and schedules to take automated snapshot backups. It also has a Windows agent to consistently back up Windows applications. CPM allows you to recover a volume from a snapshot, increase its size and switch it with an existing attached volume, in a single step. To see the different editions with pricing and details, refer to N2Ws’ pricing page on our website.