EC2 snapshots (or EBS snapshots) are the best means to create an EC2 backup solution. Although most EC2 management solutions offer little in the way of managing snapshots, you can use a comprehensive solution like N2WS, to manage your backup operations. But why are EC2 snapshots the best method? For that we need to look into the advantages of snapshot technology over traditional file-level backup solutions.
EC2 snapshots are not different in essence from hardware snapshots that storage devices offer in traditional datacentres. When we refer to “snapshot” we usually mean a block-level incremental snapshot. This means that every snapshot that is taken only records the disk-level changes made to the disk since the last snapshot. This method is extremely effective and efficient because:
- Data Reduction – The data recorded in every snapshot is minimal, which helps in terms of storage resources as well as computation and networking resources. In EC2 it directly affects your AWS bill.
- Snapshots are taken in the storage device itself and therefore do not affect the host. This property together with the previous one allows taking snapshots at a much more frequent rate than traditional file-level backup solutions, which occur in the host and consume host resources.
- The most powerful feature of snapshots is that their block-level structure allows them to be recovered instantly, by performing a “lazy” background recovery.
The main advantage of snapshots is probably the fact that they happen instantly. In fact, this is not entirely true; taking a snapshot can take time. EC2 snapshots are copied to S3, and it takes time to copy data, even if these are only incremental changes.
So, how does it work? We don’t have a view into AWS’s code, but a safe assumption would be that they are using a mechanism called “Copy on Write” or COW. The COW mechanism makes sure that the snapshot data will represent the exact image of the volume at the point-in-time of the snapshot, which is the moment it started. What it does is monitor any write requests to this volume (or disk), and starts copying the data in the background.
Whenever a write request is made to the volume, the COW mechanism checks if the target locations of this write request (i.e. the sectors on the volume) is data that the snapshot needs and has not yet copied. If this is the case, the COW mechanism will delay the write request, and copy the data out of this location, before allowing the write request to complete.
Another advantage of snapshots is when you need to backup applications consistently. For this purpose you typically need to tell the application that it is about to be backed up. Then the application closes files, flashed queues etc… and freezes waiting to be backed up. It is then essential to release (or “thaw”) the application as soon as possible, because keeping it frozen for long will result in failed requests and a disruption of operations.
The “instantness” of the snapshots allows this frozen state to last only a very short time (usually a fraction of a second) since it only needs to wait for the snapshot to start. Right after the snapshots starts, the COW mechanism will make sure the snapshot remains consistent, regardless of how long the actual copy of the snapshot will take.
Operational backup, as opposed to long-term backup or archiving, is typically used when there’s an outage or another data loss event. Operational backup is usually short term and the key is to have the most recent copy of the data (i.e. backup) possible and to be able to recover it quickly to minimize downtime. With operational backup you will need very fast backup and recovery, and the best way to achieve that is by using snapshots.
When planning how to deal with the various EC2 management issues, the best answer to “how to back up my instances” question is by using EBS snapshots. That said, you will need a backup solution like N2WS to take care of other needs that a comprehensive backup solution needs to fulfill. Among these needs are: Automation and scheduling, management and monitoring, application consistency, retention management, easy recovery options.
