fbpx
How to Automate EC2 Instance Backup

How to Automate EC2 Instance Backup Step-by-Step

how to automate ec2

As your organization grows, your data grows. And that growing data is generated from a growing number of endpoints (like desktops, laptops, servers, virtual machines, and other devices).

That’s why automating your backup solution isn’t just time-saving, it’s very cost effective, too. But, trying to leverage non-cloud-native solutions to backup AWS resources may not be effective in achieving this desired resource-saving benefit.

That’s because traditional backup software and methods are very centralized by nature —creating innate disadvantages, such as single points of failure, high-cost software licensing, and requiring dedicated hardware resources. So, in this article, we’ll walk through how to backup your EC2 instances using cloud-native technologies —EBS snapshots and AMIs— to achieve both cost-saving and time-saving benefits while maximizing your data availability.

2 Different Ways to Take EC2 Instance Backups

There are two ways to implement backups of your EC2 instances on AWS:

  • If your instance is EBS-backed, you can create the snapshots of the EBS volume.
  • Or, you can create an AMI of your instances as a backup solution.

Both the EBS snapshot and the AMI are automatically stored on Amazon S3 which is known for being highly durable and reliable. (Learn more with this post: 3 Ways To Backup And Recover EC2 Windows Instances)

Option 1: Manual EC2 Instance Backup by an EBS Snapshot

Before showing how to automate your instance backup, let’s first look at the different manual steps we should perform. Follow the steps below to implement a backup for an EC2 EBS volume:

  • Open AWS Console.
  • Click the “Instances” section in AWS console under the EC2 dashboard.

how to automate ec2

    • Select the instance for which you want to create a backup.
    • Under the description tab for that instance, you can see the details of that instance which will also show you the block devices. Clicking on a block device will show the volume ID

how to automate ec2 volume_id

    • Click on EBS ID (volume ID), which will take you to the volume section under the EC2 dashboard. To create the snapshot click on the action button and select “Create Snapshot” option.

create_snapshotA

    • Enter a name and description for the AWS snapshot in the snapshot dialog as below. After creating a snapshot you can see your AWS snapshot in the snapshot section under the EC2 dashboard.

how to automate ec2 To restore the volume from an AWS snapshot:

  • Search for the snapshot under the snapshot section using its description.
  • Right-click and select the “Create Volume” option.
  • Fill in the required details in the Create Volume dialog box and click the “Create” option.
  • A volume with the same snapshot will be created and then you can attach the new volume to an EC2 Instance for further use.

how to automate ec2

Option 2: Manual Backup Using AMI

This solution can be used if your instance is not EBS backed up. To create the EC2 Instance backup, you can create the AMI of an Instance. Furthermore, by using the same AMI you can launch an instance in the same state.

  • Search for the Instance for which you want to create the AMI. Select the “Create Image” option under Image after clicking on the action button.

how to automate ec2

  • Follow the steps below once you have selected the “Create Image” option:
    • Enter the AMI details in the “Create Image” dialog box as below.
    • Enter the AMI Name and Description.
    • If you want your instance to stay in a running state without restarting while creating the AMI, then check the “No Reboot” option.
    • You can also customize the size and other options like “delete on termination”. The “delete on termination” option will delete the volume if the instance is terminated, otherwise, it will still be in an available state if the EC2 instance is terminated.

how to automate ec2

The Better Option: Automating the EC2 Backup

To automate the EC2 Backup, you will need to write a script to automate the above steps by using AWS’ API. Below is the step by step process which should be followed in the script:

  • Get the list of instances.
  • Connect to AWS through API to list the Amazon EBS volumes that are attached locally to the instance.
  • List the snapshots of each volume.
  • Assign a retention period to the snapshot.
  • Create an AWS snapshot of each volume.
  • Delete the AWS snapshot if it is older than the retention period.

By using AWS Command Line Interface (AWS CLI) you can write a shell script which will be used for automating the EBS volume backup. It’s recommended to install the AWS CLI if it has not already been installed. You can refer to this resource for details:AWS CLI Installation.

Commands to Install AWS CLI

curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
unzip awscli-bundle.zip
./awscli-bundle/install -b ~/bin/aws

how to automate ec2 After installing AWS CLI, configure it using the aws configure command

aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: ENTER

Creating Shell Script

Use the script below to copy code to snapshot.sh and set it as cron for automatic timely backup. You can find an explanation of script in comments.

#!/bin/bash
# Volume list file will have volume-id:Volume-name format
VOLUMES_LIST = /var/log/volumes-list
SNAPSHOT_INFO = /var/log/snapshot_info
DATE = `date +%Y-%m-%d`
REGION = "eu-west-1"
# Snapshots Retention Period for each volume snapshot
RETENTION=6
SNAP_CREATION = /var/log/snap_creation
SNAP_DELETION = /var/log/snap_deletion
EMAIL_LIST = abc@domain.com
echo "List of Snapshots Creation Status" > $SNAP_CREATION
echo "List of Snapshots Deletion Status" > $SNAP_DELETION
# Check whether the volumes list file is available or not?
if [ -f $VOLUMES_LIST ]; then
# Creating Snapshot for each volume using for loop
for VOL_INFO in `cat $VOLUMES_LIST`
do
# Getting the Volume ID and Volume Name into the Separate Variables.
VOL_ID = `echo $VOL_INFO | awk -F":" '{print $1}'`
VOL_NAME = `echo $VOL_INFO | awk -F":" '{print $2}'`
# Creating the Snapshot of the Volumes with Proper Description.
DESCRIPTION = "${VOL_NAME}_${DATE}"
/usr/local/bin/aws ec2 create-snapshot --volume-id $VOL_ID --description "$DESCRIPTION" --region $REGION &>> $SNAP_CREATION
done
else
echo "Volumes list file is not available : $VOLUMES_LIST Exiting." | mail -s "Snapshots Creation Status" $EMAIL_LIST
exit 1
fi
echo >> $SNAP_CREATION
echo >> $SNAP_CREATION
# Deleting the Snapshots which are 10 days old.
for VOL_INFO in `cat $VOLUMES_LIST`
do
# Getting the Volume ID and Volume Name into the Separate Variables.
VOL_ID = `echo $VOL_INFO | awk -F":" '{print $1}'`
VOL_NAME = `echo $VOL_INFO | awk -F":" '{print $2}'`
# Getting the Snapshot details of each volume.
/usr/local/bin/aws ec2 describe-snapshots --query Snapshots[*].[SnapshotId,VolumeId,Description,StartTime] --output text --filters "Name=status,Values=completed" "Name=volume-id,Values=$VOL_ID" | grep -v "CreateImage" > $SNAPSHOT_INFO
# Snapshots Retention Period Checking and if it crosses delete them.
while read SNAP_INFO
do
SNAP_ID=`echo $SNAP_INFO | awk '{print $1}'`
echo $SNAP_ID
SNAP_DATE=`echo $SNAP_INFO | awk '{print $4}' | awk -F"T" '{print $1}'`
echo $SNAP_DATE
# Getting the no.of days difference between a snapshot and present day.
RETENTION_DIFF = `echo $(($(($(date -d "$DATE" "+%s") - $(date -d "$SNAP_DATE" "+%s"))) / 86400))`
echo $RETENTION_DIFF
# Deleting the Snapshots which are older than the Retention Period
if [ $RETENTION -lt $RETENTION_DIFF ];
then
/usr/local/bin/aws ec2 delete-snapshot --snapshot-id $SNAP_ID --region $REGION --output text> /tmp/snap_del
echo DELETING $SNAP_INFO >> $SNAP_DELETION
fi
done < $SNAPSHOT_INFO
done
echo >> $SNAP_DELETION
# Merging the Snap Creation and Deletion Data
cat $SNAP_CREATION $SNAP_DELETION > /var/log/mail_report
# Sending the mail Update
cat /var/log/mail_report | mail -s "Volume Snapshots Status" $EMAIL_LIST

Follow the steps below for creating and running shell script:

  • Create a script by the name of snapshot.sh using the command below.
  • Set it as a cron in crontab.
# Edit Cron File
crontab -e

Taking a backup of your infrastructure resources frequently is very important in order to be able to recover from a disaster. It’s important to schedule AWS backups on a timely basis, such as taking backup weekly or monthly on different availability zones. It’s one of the best practices that is followed by DevOps teams all over the world.

Automating Instance Backup Using N2WS Backup & Recovery

While using in-house scripts can provide for a basic backup solution, it doesn’t make business sense for organizations to invest in a fully-featured in-house backup solution rather than focusing on their business-critical tasks. That’s why our founder Uri created N2WS Backup & Recovery. It’s an enterprise-class backup-recovery and disaster recovery solution designed for AWS EC2 covering all the essential backup and recovery features to ensure the robustness of your data as well as simplifying processes and saving precious DevOps time.

N2WS Backup & Recovery is available as a service model that allows users to manage multiple AWS accounts and configure policies and schedules to take automated snapshot backups. It also has a Windows agent to consistently back up Windows applications without the need for maintenance windows. Furthermore, in a dynamic cloud environment, you need to be able to keep consistent backup policies across all your instances at any point in time.

To be most effective, your solution needs to be dynamic and automated when a server is terminated and a new instance needs to be launched. Using EC2 tags, N2WS can automatically assign each one of these new instances the appropriate backup policy based on their purpose and your initial configuration. For more info, check out our previous article about tag-based continuous AWS cloud backup. And try N2WS Backup & Recovery for FREE!

Keep Reading:

Share this post →

Share on linkedin
Share on twitter
Share on facebook
Share on email
You might also like: