Backup AWS Ephemeral Storage of an EC2 Instance: Part 2

Backup Ephemeral Storage of an EC2 Instance: Part 2In Part 1 of this article, we introduced you to the concept of ephemeral storage and its use case. We discussed that sometimes it’s possible to configure a database on instance store-backed storage for faster performance. In Part 2, we will show you how to back up data stored on instance store-backed storage to an AWS EBS volume. In this part, we will launch an EC2 backup instance with instance store-backed root volume. We will install MySQL on the ephemeral storage and setup for automated SQL server backup using CPM. We have provided a step-by-step guide for setting up and performing these functions below.

 

Step 1: Creating an EC2 Instance with Instance Store Volume Attached

Generally, the instance store AMIs are available under the Custom AMIs section in EC2 launch wizard. Find a suitable AMI and launch an instance with it.

Diagram of ephemeral storage step - find your AMI

 

First, select the instance type.

Diagram of ephemeral storage step - selecting an instance type

 

The storage type will be Instance Store.

Diagram of ephemeral storage step - adding instance storage

 

Note: You are not required to specify the size of instance store volume. The size of instance storage is decided based on instance type.

Then, launch the instance.

Diagram of ephemeral storage step - launching an instance

 

Step 2: Setting up the Database on an EC2 Instance

Set up a MySQL database on an EC2 instance.

 

Step 3: Create and Attach an EBS Volume for Persistent Backup

In this step, we will create an EBS volume for backup of MySQL dumps, which will need to be attached to our EC2 instance. After you create the EBS volume, attach it to the EC2 instance.

Diagram of ephemeral storage step - your EC2 instance

 

Next, list storage devices attached to the instance.

Diagram of ephemeral storage step - list of storage

 

After that, change the file system format of the EBS volume. You will first need to create a partition table, as shown below.

Diagram of ephemeral storage step - EBS volume

 

Once the partition table is ready, you can format the volume of it using the ext4 file system.

Diagram of ephemeral storage step - Partition Table

 

Then, create a mount directory where you want to mount the device using the code

mkdir /home/ec2-user/backup-volume

Open the /etc/fstab file and add a new line at the end of the file.

/dev/xvdf1 /home/ec2-user/backup-volume ext4 defaults,nofail 0 2

This step is important, as it will ensure that the EBS volume is attached automatically on instance restart. Once you have added the new line, save and run the

sudo mount -a

command to execute changes in the /etc/fstab file.

 

Step 4: Create a Script That Will Take MySQL Dump and Store It in EBS Volume

This script is to be created on instance-store instance’s home directory /home/ec2-user. The motive of this script is to take backups directly to the EBS volume that was mounted in the previous steps.

Create a file named db-backup.sh and copy and paste the following the content in it:

#!/bin/sh

# DB credentials
mysqluser=""
mysqlpassword=""
dbname="test"

now="$(date +'%d_%m_%Y_%H_%M_%S')"
filename="db_backup_$now".gz

#Backup location
backupfolder="/home/ec2-user/backup-volume"
fullpathbackupfile="$backupfolder/$filename"

#Log file location
logfile="$backupfolder/"backup_log_"$(date +'%Y_%m')".txt

#Remove old backups
find $backupfolder -type f -name "*.gz"  -exec rm {} \; >> "$logfile"
echo "deleted all files,operation finished at $(date +'%d-%m-%Y %H:%M:%S')" >> "$logfile"

#Take mysql dump
echo "mysqldump started at $(date +'%d-%m-%Y %H:%M:%S')" >> "$logfile"
mysqldump --user="$mysqluser" --password="mysqlpassword" --default-character-set=utf8 "$dbname" | gzip > "$fullpathbackupfile"
echo "mysqldump finished at $(date +'%d-%m-%Y %H:%M:%S')" >> "$logfile"
echo "*****************" >> "$logfile"
exit 0

Please note that you will be required to provide a mysql username, mysql password, and database name in the code above before executing it. The “before script” will execute the script shown above and will take MySQL data backup and store it on the EBS volume before the EBS volume backup process is executed.

 

Step 5: Create the “Before Script” on the CPM Instance

The “before script” is launched by the CPM before the EBS snapshot is taken. This script is typically used to move applications into backup mode. It is also commonly used to copy and transfer data onto a disk or to set up a backup target before a snapshot is taken. This state will be maintained for a very short time—only until the snapshots of the policy are fired.

Here we will use the db-backup.sh script that we created earlier in a “before script” to take backup. We will use the script created in step 4 to take MySQL DB dump and store it directly into the AWS EBS volume. This script will be executed by the “before script” of the CPM and it will perform the following steps:

  1. SSH onto CPM server with user cpmuser.
  2. Navigate to /cpmdata/scripts.
  3. Create a directory name with your username. In this article, we are using “admin” as the user, so we made the directory name “admin.”
  4. Create “before script” in /cpmdata/scripts/admin. The naming convention for the “before script” is “before_<Policy Name>.sh”. For example, add the code below to the before_Backuppolicy.sh file:
#!/bin/bash

ssh -i <location of pem file to SSH>  <user-name>@<IP Address EC2 instance where your rsync backup script> "db-backup.sh"

Diagram of ephemeral storage step - the "before script"

 

The code above will SSH into the EC2 instance and execute the backup script before the actual backup policy is executed.

This code will execute the db-backup.sh script on our target EC2 instance where we’ve attached the EBS for backup of dumps.

The /cpmdata/scripts should look like this with proper permissions and file ownership:

Diagram of ephemeral storage step - script permissions

 

Step 6: Configure a CPM Automated Backup

To configure Auto Backup, create a policy in the CPM and configure it to run with the configurations shown in the picture below:

Diagram of ephemeral storage step - automatic backup policy

 

After you’ve created the policy, configure the CPM server instance to execute this policy after the “before script.” The “before script” was created in step 5.

Once this has been done, we are ready to execute the backup policy and see if it’s able to execute the “before script” as intended. The last thing to do is to check if the snapshot was taken properly. You can view the output of the log file and console to verify this.

Diagram of ephemeral storage step - checking your policy snapshot

 

By following all of these steps, you’ve successfully managed to back up MySQL data from an instance store-backed storage to a persistent EBS volume with the help of CPM manager and “before scripts.”

 

Conclusion

Instances with ephemeral storage can be scaled horizontally when they are under load balancing and part of autoscaling, but they can’t be scaled vertically for disk storage. You are also unable to change configurations like RAM or CPU once these instances are launched. This is the reason why ephemeral storage should be used for special use cases once you have a thorough understanding of your business requirements.Even though instance store is a temporary storage and EBS is persistent storage, you should never rely only on EBS storage for backup alone. Although the chances are slim, it is possible for data to be corrupted or lost on an EBS volume due to an internal issue or disk corruption. We recommend performing consistent backups of EBS volume using snapshot to avoid any data loss.

Even though instance store is a temporary storage and EBS is persistent storage, you should never rely only on EBS storage for backup alone. Although the chances are slim, it is possible for data to be corrupted or lost on an EBS volume due to an internal issue or disk corruption. We recommend performing consistent backups of EBS volume using snapshot to avoid any data loss.

In this article, we showed how N2W Software’s Cloud Protection Manager can help you set up an EC2 automated backup of your instance storage MySQL data to EBS storage. CPM offers options for before, after, and completed scripts that help achieve application-consistent backup. The “before script” is useful to execute a specific function before a backup policy is executed, as this way one can set up to take the backup of MySQL data from ephemeral storage to EBS volume before the creation of the EBS volume snapshot.

CPM is an enterprise-class backup, recovery, and disaster recovery solution for AWS EC2. It uses existing AWS EBS volume and AWS RDS database snapshot abilities to automatically take snapshots at regular intervals. Additionally, you can set up policies and schedule backups for various targets. CPM helps manage snapshots with policies. For example, if you have multiple snapshots, the older ones may be irrelevant. With CPM, you can configure a policy to delete the snapshots after a certain period. This also helps in cost savings and effective backup management. The safety and security of your data are the most important aspects of your deployment on the cloud—learn more about how to protect it with Cloud Protection Manager.

 

Share this post →

You might also like: