How To Sync Data From AWS EFS To AWS EBS Using N2WS

Amazon Elastic File System (AWS EFS) is one of the most popular storage options available, able to scale both capacity and performance automatically without manual intervention or any upfront payments. Amazon EFS is a perfect fit as a managed network file system, allowing file systems to be shared across different AWS EC2 instances.

It has an elastic capacity for growing and shrinking that enables it to act as a centralized storage solution for a broad spectrum of use cases such as web and content serving, enterprise applications, media processing workflows, container storage, big data and analytics applications, and more.

Amazon Elastic File System Overview

Since AWS EFS came out of preview mode, it has become the ideal choice for AWS customers who are looking for a shared storage solution. Amazon EFS is ideal for file storage with AWS EC2 instances because AWS EFS can be used to create file systems, mount them on Amazon EC2 instances, and then read and write data to and from those file systems. You can mount an AWS EFS file system in your VPC, through the Network File System versions 4.0 and 4.1 (NFSv4) protocol. Amazon EFS is ideal for use cases like:

• A storage solution which can grow and shrink in accordance with the usage
• A file system that can be shared between instances
• An easy to setup, resilient and secure file storage solution
• A highly available storage solution which can redundantly store objects across availability zones
• A high-performance file system that can store and serve information for a range of web applications

We know there are advantages to using Amazon Elastic Block Store (AWS EBS) for file systems, but AWS EBS can only be accessed by a single instance at a time. When you require a storage system that can be accessed by multiple instances, you will need to use AWS EFS. While AWS EFS offers security, redundancy, and scalability, there are also cases where customers will want to back up their data to a persistent and isolated storage place.

AWS EBS is a perfect solution for this kind of backup as it has the ability to create application-consistent snapshots. In this article, we will show you how to synchronously backup data from EFS to EBS using N2WS Backup & Recovery. With this solution in place, even if someone unknowingly deletes your data from AWS EFS, you will always have a backup available which can be restored.

Backing Up AWS EFS Data to an AWS EBS Volume Using N2WS:

This guide will show you how to backup AWS EFS data to an AWS EBS volume. Before you begin, here is an overview of the action items:

1. Creating an AWS EC2 instance with an AWS EFS attached.
2. Setting up a web server with a demo website, where all the files related to the website will be mounted on EFS volume.
3. Mounting an AWS EBS volume to the same AWS EC2 instance.
4. Writing a “before script” that uses the

   rsync

   command to copy data from AWS EFS to AWS EBS.

5. Setting up an automated AWS EBS Volume snapshot by using N2WS Backup & Recovery.

Step One: Set Up an Apache Web Server on an EC2 Instance

1. The first task will be to set up an Apache web server on an EC2 instance. You will use Apache HTTP Server to serve files stored on your Amazon EFS file system.
2. Create a directory efs-mount-point in

   /var/nas/content/live/n2ws/html

   where you will mount the AWS EFS file system.

3. Change the DocumentRoot parameter from

   /etc/httpd/conf/httpd.conf

   to

   /var/nas/content/live/n2ws/html/efs-mount-point

   and restart Apache web server.

Step Two: Attach an AWS EFS Volume

1. You will need an EFS DNS name to attach it to the EC2 instance. If you don’t have one, follow this document to launch a new EFS volume.
2. Once you have a DNS name available for the EFS volume, on your instance create a directory in
   

   /var/nas/content/live/n2ws/html

   
   

   where you will mount the EFS volume:

   
   

   sudo mkdir /var/nas/content/live/n2ws/html/efs-mount-point

   
   
3. The next step will be to mount EFS volume to:
   

   /var/nas/content/live/n2ws/html/efs-mount-point

   
   

   sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,
   hard,timeo=600,retrans=2 fs-58d2324.efs.us-west-2.amazonaws.com:/
   /var/nas/content/live/n2ws/html/efs-mount-point

   
   

Step Three: Create and Attach AWS EBS for Backup

In this step, you will create an AWS EBS volume for backup. This needs to be attached to an AWS EC2 instance.

1. After creating an AWS EBS volume, attach it to an AWS EC2 instance: 
2. Check the AWS EBS volume’s listing on the instance: 
3. Since it is a raw unformatted volume, you will have to format it and create a file system on AWS EBS volume. First, you need to create a partition table:

   sudo fdisk /sev/xvdf

   

4. Once the partition table is ready you can format the volume with an

   ext4

   

   file system: 

5. Create a mount directory where you want to mount the device:

   mkdir /home/ec2-user/backup-volume

6. Open the

   /etc/fstab

   file and add a new line at the end of file. In this way you would logically map the AWS EBS volume

   /dev/xvdf1

   to the local directory named

   /home/ec2-user/backup-volume

   whenever the system reboots:

   /dev/xvdf1 /home/ec2-user/backup-volume ext4 defaults,nofail 0 2

   Once added, save and run the following command to execute changes in the

   /ets/fstab file

   sudo mount -a

Step Four: Creating a “Before Script” on a N2WS Instance

The “before script” is launched before AWS EBS snapshots are taken. Typically, this script is used to move applications to backup mode. The “before script” usually leaves the system in a frozen state. The system will only remain in that state shortly until the EBS snapshots of the policy are fired. Here we will use the

rsync

command to copy data from an AWS EFS to an AWS EBS. We will create a simple script which will sync data from

/var/nas/content/live/n2ws/html/efs-mount-point to /home/ec2-user/backup-volume

. This script will be used as the “before script” by N2WS later:

1. SSH login onto the N2WS server with user

   "cpmuser”

2. Navigate to

   /cpmdata/scripts

   

3. Create a directory. We created a directory using our username. In this article we are using

   “admin”

   username, hence create directory with name

   "admin”

4. Create the “before script” in

   /cpmdata/scripts/admin

   . The naming convention for the “before script” is

   "before_.sh”

   which, for example, will look like

   before_Backuppolicy.sh

5. Add the code below to the

   before_BackupPolicy.sh file:#!/bin/bash ssh -i   @ "bash volume-backup.sh"

   

Note: The above code will transform SSH into an AWS EC2 instance and execute the backup script before the actual backup policy is executed.

Step Five: Configure a N2WS Backup & Recovery Automated Backup

To configure an automated backup, create a policy in N2WS and configure it to run as per the below configurations:  Once you’ve created the policy, configure the N2WS server instance to execute this policy after the “before script.” Now set up a “before script” which will execute a volume backup script before taking a snapshot.

1. SSH login onto the N2WS server with the username

   "cpmuser”

2. Navigate to

   /cpmdata/scripts

3. Create a file named

   "before_”

   next. For example, we would create file named

   “before_BackupPolicy.sh”

4. In the file above, append the following code:  This code will execute the

   volume-backup.sh

   script on our target AWS EC2 instance where we’ve attached AWS EFS and AWS EBS for backup.

5. The

   volume-backup.sh

   script will look like this:

   #! /bin/bash Echo “Please wait, backup started” Rsync -rtvu --delete /var/nas/content/live/n2ws/html/efs-mount-point/ /home/ec2-user/backup-volume Echo “File synced!”

   

   You will create backup on AWS EC2 instance which has an AWS EFS and an AWS EBS volume attached 

6. With proper permissions and file ownership, the

   /cpmdata/

   scripts should look like this: Once this is done, it is time to execute the backup policy and see if it is able to execute the “before script” as intended. The last thing to do is to check that the snapshot was successful. You can check the output of the log file and console to verify. If you’ve followed all these steps correctly, you have successfully managed to backup AWS EFS data to a persistent AWS EBS volume with the help of N2WS and “before scripts.”

Conclusion

AWS EFS is a one-of-a-kind utility that allows your storage to grow according to your needs in the cloud. Although AWS EFS costs are ten times higher than the costs of using AWS EBS, it still is a good option for a network file storage solution as it is secure, flexible, and redundant.

In this article, we showed how N2WS Backup & Recovery can help you set up an automated backup of your AWS EFS files. N2WS offers options for before, after, and completed scripts that help achieve application-consistent backup. The “before script” is useful to execute a specific function before a backup policy is executed, as this way one can set up checks to ensure that data is copied correctly while IO is frozen during the creation of a snapshot of EBS volumes.