N2WS AWS S3 Replication – Using N2WS S3 Replication to Synchronize Your Crucial Data

S3 Replication
Share This Post

Ten years ago, most businesses relied entirely on their legacy bare metal hardware and had little understanding of what the cloud was. Today, it’s quite difficult to find a company that hasn’t at least experimented with a hybrid architecture that includes one of the public cloud providers. Cloud computing has rooted itself into modern IT culture—so much so that it has become an industry standard.

Even though public cloud providers like Amazon have created and made available a wide array of highly effective services and tools, there’s always room for improvements to their offerings. As a result, there are a multitude of third-party products designed to help you work with AWS. One such product is N2WS Backup and Recovery, a tool that was developed to centralize your AWS cloud backups. N2WS Backup and Recovery has just undergone an update (3.1) which has added a host of new features. This article will take a brief look at those features before exploring S3 Replication, one the most useful of these additions, in greater depth.

N2WS 3.1: New Features

In March of this year, N2WS Backup and Recovery 3.0 was released. In it, some great features were introduced, including a completely new user interface with a new dashboard, a cost explorer, recovery scenarios, resource control cost savings, and policy time zones.

N2WS v3.1 was recently released and can be accessed via the AWS Marketplace. The following features await you in this new version:

  • Optimized N2WS Worker Instances—By using the newly-released AWS EBS Snapshot API, N2WS has improved its mechanisms for copying to Amazon S3 and archiving to Amazon Glacier. This long and somewhat expensive process can now be done more cheaply, since this new mechanism can read the AWS EBS snapshot directly and capture metadata. The optimized worker instances now ensure much faster transfers, speeding up the entire process significantly.
  • EBS Snapshot Reports—Thanks to the AWS EBS Snapshot API, N2WS now enables you to produce a report that shows all of the metadata related to a particular snapshot. This metadata includes details like size of EBS volumes (actual volume size), valid data size (the part of that volume used to store the data), and changed data size (incremental change for subsequent snapshots).
  • N2WS ZeroEBS™—This new feature allows you to eliminate the need to retain the EBS snapshot after you’ve copied data to Amazon S3 or Glacier. Now, after the task is complete, the original EBS volume can be removed by ticking a box that appears when you create a policy. The feature helps customers to control their spending.
  • Recovery Scenarios Scheduling—Originally introduced in the 3.0 update, Recovery Scenarios offers you the option to perform bulk recoveries (recovering multiple machines at the same time). The scheduling feature introduced in version 3.1 improves on this offering by allowing you to define the order in which targets will be recovered and granting you complete control over the automation of these tasks.
  • EFS Cross-Region Disaster Recovery—EFS Disaster Recovery has been around since the 2.6 update; however, with version 3.1, you can now backup and recover your EFS volumes across multiple AWS regions. This is crucial for ensuring a proper AWS disaster recovery plan, since your primary region might not be the best place to store backups.
  • Scan Tags with Custom Tags—Tag-based backups are incredibly useful because they allow you to easily automate resource backups. You can even group the backup processes for various different services (e.g., EC2, RDS, EFS, and DynamoDB) by using tags. Up until now, you could only accomplish this by using the tag “cpm backup.” Allowing clients to choose their own tags is important, and this new feature allows you to do just that.
  • Amazon S3 Replication—With Amazon S3 Replication, clients can now synchronize and replicate static data between two different AWS S3 buckets. This replication can take place between two buckets within the same AWS region, between two completely different regions, or even between two different AWS accounts. This feature provides a powerful layer of additional security for a wide variety of scenarios, such as accidental deletion and outage events which can affect your primary region. Malicious attacks can affect your entire AWS account, so having your crucial data stored safely on another account can be what saves your business. Given the industry’s current emphasis on compliance, having an “off-site” backup storage solution can be extremely valuable.

N2WS 3.1: A How-to Guide to S3 Replication

In previous articles, we’ve covered in detail how to provision and set up your N2WS EC2 instance. This article will focus on the S3 Replication procedure.

When you first open your updated version of N2WS Backup and Recovery, you’ll be greeted with the new dashboard. Because we’re using a trial account in this example, it looks completely new.

Main Dashboard of N2WS v3.1

To start S3 Replication, go to the Policies section:

Here, we’ll create a new policy for this example.

Name your policy, and choose the N2WS user and account to be used for this policy (in case you have multiple ones). If you already have a predefined schedule for this task, you can specify it here. In this example, we’ll trigger the replication job manually.

Create a policy

Now, go to the “Backup Targets” tab, and click on “Add Backup Targets.”

The last option is what we’re looking for, so click on “S3 Bucket Sync.”

When the window pops up, choose the bucket you want to replicate. Here, we’ll use our “n2ws-s3-repo” bucket.

Choose what bucket to replicate

After the bucket is added, start configuring the replications by clicking on the word “Configure.”

Here’s what the configuration looks like:

Policy S3 Bucket Sync configuraiton

We’re going to leave the source prefix (or the path we want to use) empty so that the entire bucket is synced. Note that if you choose a specific prefix, you can tick the option to keep the source prefix at destination. This can be useful when using long prefixes or when always syncing to the same location.

Under “Sync Destination,” choose the target S3 bucket where the replication will occur and a destination prefix, if needed. You can also pick the S3 Storage Class, allowing you to, for example, copy your S3 Standard tier objects as S3 Infrequent Access in order to control costs.

The last option is the “Delete Extra” box. This feature allows you to ensure that the destination bucket is always synced to the source bucket. If you check this box, any files located in the destination bucket that are not in the source bucket will be removed during the replication job. Keep in mind that there are many cases where this option should be avoided, since your destination bucket might contain other backups or files that you do want to keep there. When you’ve completed these steps, click on “Apply.” Now, all you need to do is run this task (unless you’ve chosen a predefined schedule for it) by clicking on “Run ASAP.”

As is the case with other backup tasks, you can follow the progress of the replication through the Backup Monitor. In the screenshot below, you can see that the task is being initialized:

Progress shown in backup monitor

If we go to our backup S3 bucket, we can see that it’s still empty, as shown below:

Your empty S3 Bucket

After a few seconds, the task will be performed.

Now, if we go back to our backup S3 bucket, there’s a new file in it—one that was just replicated via N2WS Backup and Recovery.

Take a few minutes today to set up your S3 Replication

After running through the features introduced in the recent N2WS v3.1 update and considered the benefits they can provide for you and your business, we then guided you through the entire process of using Amazon S3 Replication. This new features takes only minutes to set up—minutes that can be critical to the safety of your business. If you already rely on N2WS Backup and Recovery, there’s no reason not to implement this new feature. If you don’t, now is a great time to get started with this powerful tool. A free trial (no credit card needed) is available here.

Next step

The easier way to recover cloud workloads

Allowed us to save over $1 million in the management of AWS EBS snapshots...

Try N2WS for Free