How To Sync Data From AWS EFS To AWS EBS Using CPM

How To Sync Data From AWS EFS To AWS EBS Using CPM Amazon Elastic File System (AWS EFS) is one of the most popular storage options available, able to scale both capacity and performance automatically without manual intervention or any upfront payments. AWS EFS is a perfect fit as a managed network file system, allowing file systems to be shared across different AWS EC2 instances. It has an elastic capacity for growing and shrinking that enables it to act as a centralized storage solution for a broad spectrum of use cases such as web and content serving, enterprise applications, media processing workflows, container storage, big data and analytics applications, and more.

A Little Bit About EFS

Since Amazon EFS came out of preview mode, it has become the ideal choice for AWS customers who are looking for a shared storage solution. Amazon EFS is ideal for file storage with AWS EC2 instances because AWS EFS can be used to create file systems, mount them on Amazon EC2 instances, and then read and write data to and from those file systems. You can mount an Amazon EFS file system in your VPC, through the Network File System versions 4.0 and 4.1 (NFSv4) protocol.

Amazon EFS is ideal for use cases like:

  • A storage solution which can grow and shrink in accordance with the usage
  • A file system that can be shared between instances
  • An easy to setup, resilient and secure file storage solution
  • A highly available storage solution which can redundantly store objects across availability zones
  • A high-performance file system that can store and serve information for a range of web applications

We know there are advantages to using Amazon Elastic Block Store (AWS EBS) for file systems, but AWS EBS can only be accessed by a single instance at a time. When you require a storage system that can be accessed by multiple instances, you will need to use AWS EFS.

While AWS EFS offers security, redundancy, and scalability, there are also cases where customers will want to back up their data to a persistent and isolated storage place. AWS EBS is a perfect solution for this kind of backup as it has the ability to create application-consistent snapshots. In this article, we will show you how to synchronously backup data from EFS to EBS using N2W Software Cloud Protection Manager (CPM). With this solution in place, even if someone unknowingly deletes your data from AWS EFS, you will always have a backup available which can be restored.

Backing Up AWS EFS Data to an AWS EBS Volume Using CPM:

This guide will show you how to backup AWS EFS data to an AWS EBS volume. Before you begin, here is an overview of the action items:

  1. Creating an AWS EC2 instance with an AWS EFS attached.
  2. Setting up a web server with a demo website, where all the files related to the website will be mounted on EFS volume.
  3. Attaching an AWS EBS volume to the same AWS EC2 instance.
  4. Writing a “before script” that uses the
    rsync

    command to copy data from AWS EFS to AWS EBS.

  5. Setting up AWS EBS automated backup by using Cloud Protection Manager with.

Step One: Set Up an Apache Web Server on an EC2 Instance

  1. The first task will be to set up an Apache web server on an EC2 instance. You will use Apache HTTP Server to serve files stored on your Amazon EFS file system.
  2. Create a directory efs-mount-point in
    /var/nas/content/live/n2ws/html

    where you will mount the AWS EFS file system.

  3. Change the DocumentRoot parameter from
    /etc/httpd/conf/httpd.conf

    to

    /var/nas/content/live/n2ws/html/efs-mount-point

    and restart Apache web server.

Step Two: Attach an AWS EFS Volume

  1. You will need an EFS DNS name to attach it to the EC2 instance. If you don’t have one, follow this document to launch a new EFS volume.You will need an EFS DNS name to attach it to the EC2 instance.
  2. Once you have a DNS name available for the EFS volume, on your instance create a directory in
    /var/nas/content/live/n2ws/html

    where you will mount the EFS volume:

    sudo mkdir /var/nas/content/live/n2ws/html/efs-mount-point
  3. The next step will be to mount EFS volume to:
    /var/nas/content/live/n2ws/html/efs-mount-point
    sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,
    
    hard,timeo=600,retrans=2 fs-58d2324.efs.us-west-2.amazonaws.com:/
    
    /var/nas/content/live/n2ws/html/efs-mount-point

Step Three: Create and Attach AWS EBS for Backup

In this step, you will create an AWS EBS volume for backup. This needs to be attached to an AWS EC2 instance.

  1. After creating an AWS EBS volume, attach it to an AWS EC2 instance: After creating an AWS EBS volume, attach it to an AWS EC2 instance
  2. Check the AWS EBS volume’s listing on the instance: Check the AWS EBS volume’s listing on the instance
  3. Since it is a raw unformatted volume, you will have to format it and create a file system on AWS EBS volume. First, you need to create a partition table:
    sudo fdisk /sev/xvdf

    create a partition table

  4. Once the partition table is ready you can format the volume with an
    ext4

    file system: format the volume with an ext4 file system

  5. Create a mount directory where you want to mount the device:
    mkdir /home/ec2-user/backup-volume
  6. Open the
    /etc/fstab

    file and add a new line at the end of file. In this way you would logically map the AWS EBS volume

    /dev/xvdf1

    to the local directory named

    /home/ec2-user/backup-volume

    whenever the system reboots:

    /dev/xvdf1 /home/ec2-user/backup-volume ext4 defaults,nofail 0 2

    Once added, save and run the following command to execute changes in the

    /ets/fstab file

    :

    sudo mount -a

Step Four: Creating a “Before Script” on a CPM Instance

The “before script” is launched before AWS EBS snapshots are taken. Typically, this script is used to move applications to backup mode. The “before script” usually leaves the system in a frozen state. The system will only remain in that state shortly until the snapshots of the policy are fired.

Here we will use the

rsync

command to copy data from an AWS EFS to an AWS EBS. We will create a simple script which will sync data from

/var/nas/content/live/n2ws/html/efs-mount-point to /home/ec2-user/backup-volume

. This script will be used as the “before script” by CPM later:

  1. SSH login onto the CPM server with user
    "cpmuser”
  2. Navigate to
    /cpmdata/scripts

    Navigate to /cpmdata/scripts

  3. Create a directory. We created a directory using our username. In this article we are using
    “admin”

    username, hence create directory with name

    "admin”
  4. Create the “before script” in
    /cpmdata/scripts/admin

    . The naming convention for the “before script” is

    "before_.sh”

    which, for example, will look like

    before_Backuppolicy.sh
  5. Add the code below to the
    before_BackupPolicy.sh file:#!/bin/bash
    ssh -i   @ "bash volume-backup.sh"

    transform SSH into an AWS EC2 instance and execute the backup script

Note: The above code will transform SSH into an AWS EC2 instance and execute the backup script before the actual backup policy is executed.

Step Five: Configure a CPM Automated Backup

To configure an automated backup, create a policy in CPM and configure it to run as per the below configurations: configure an automated backup, create a policy in CPM

Once you’ve created the policy, configure the CPM server instance to execute this policy after the “before script.” Now set up a “before script” which will execute a volume backup script before taking a snapshot.

  1. SSH login onto the CPM server with the username
    "cpmuser”
  2. Navigate to
    /cpmdata/scripts
  3. Create a file named
    "before_”

    next. For example, we would create file named

    “before_BackupPolicy.sh”
  4. In the file above, append the following code: execute the volume-backup.sh script on our target AWS EC2 instance
    This code will execute the

    volume-backup.sh

    script on our target AWS EC2 instance where we’ve attached AWS EFS and AWS EBS for backup.

  5. The
    volume-backup.sh

    script will look like this:

    #! /bin/bash
    Echo “Please wait, backup started”
    Rsync -rtvu --delete /var/nas/content/live/n2ws/html/efs-mount-point/ 
    /home/ec2-user/backup-volume
    Echo “File synced!”

    You will create backup on AWS EC2 instance which has an AWS EFS and an AWS EBS volume attached create backup on AWS EC2 instance which has an AWS EFS and an AWS EBS volume attached

  6. With proper permissions and file ownership, the
    /cpmdata/

    scripts should look like this:
    With proper permissions and file ownership, the /cpmdata/ scripts should look like this

    Once this is done, it is time to execute the backup policy and see if it is able to execute the “before script” as intended. The last thing to do is to check that the snapshot was successful. You can check the output of the log file and console to verify.
    check the output of the log file and console to verify the snapshot was successful

    If you’ve followed all these steps correctly, you have successfully managed to backup AWS EFS data to a persistent AWS EBS volume with the help of CPM and “before scripts.”

Conclusion

AWS EFS is a one-of-a-kind utility that allows your storage to grow according to your needs in the cloud. Although AWS EFS costs are ten times higher than the costs of using AWS EBS, it still is a good option for a network file storage solution as it is secure, flexible, and redundant.

In this article, we showed how N2W Software’s Cloud Protection Manager can help you set up an automated backup of your AWS EFS files. CPM offers options for before, after, and completed scripts that help achieve application-consistent backup. The “before script” is useful to execute a specific function before a backup policy is executed, as this way one can set up checks to ensure that data is copied correctly while IO is frozen during the creation of the EBS volume snapshot.

CPM is an enterprise-class backup, recovery, and disaster recovery solution for AWS EC2. It uses existing AWS EBS volume and AWS RDS database snapshot abilities to automatically take snapshots at regular intervals. Additionally, you can set up policies and schedule backups for various targets. CPM helps manage snapshots with policies. For example, if you have multiple snapshots, the older ones may be irrelevant. With CPM, you can configure a policy to delete the snapshots after a certain period. This also helps in cost savings and effective backup management. The safety and security of your data are the most important aspects of your deployment on the cloud—learn more about how to protect it with Cloud Protection Manager.

Share this post →

You might also like: