- 13 – Tag-based Backup Management
Cloud and specifically AWS, is an environment based largely on automation. Since all the functionality is available via an API, scripts can be used to deploy and manage applications, servers and complete environments. There are very popular tools available to help with configuring and deploying these environments, like Chef and Puppet.
CPM allows configuring backup using automation tools by utilizing AWS tags. By tagging a resource (EC2 instance, EBS volume, RDS instance, Aurora Cluster or Redshift cluster), CPM can be notified what to do with this resource, and there is no need to use the GUI. Since tagging is a basic functionality of AWS, it can be easily done via the API and scripts.
To automate backup management for a resource, you can add a tag to that resource named cpm backup (lower case with a space). CPM will identify this tag and parse its content. In this tag you will be able to specify whether to:
- Remove this resource from all backup policies.
- Add the resource to a policy or list of policies.
- Create a new policy, based on an existing one (template), and then add the resource to it.
13.1.1 – Adding to a Policy or Policies
To add a resource (e.g. an EC2 instance) to an existing backup policy, all you need to do is to create the tag for this resource and specify the policy name (e.g. policy1):
- tag key: cpm backup, tag value: policy1
To add the resource to multiple policies all you need to do is to add a list of policy names, separated by spaces:
- policy1 policy2 policy3
13.1.2 – Creating a Policy from a Template
To create a new policy and to add the resource to it, add a new policy name with a name of an existing policy which will serve as a template (separated by semicolon):
- tag value: new_policy1:existing_policy1
You can also add multiple policy name pairs to create additional policies or create a policy (or policies) and to add the resource to an existing policy or policies.
When a new policy is created out of a template, it will take the following properties from it:
- Number of generations
- DR configuration
- Script/agent configuration
- Retry configuration
It will not inherit any backup targets, so you can use a real working policy as a template or an empty one.
For Script definitions:
If backup scripts are defined for the template policy, the new one will keep that definition but will not initially have any actual scripts. You are responsible to create those scripts. Since the CPM server is accessible via SSH you can automate script creation. In any case, since scripts are required, the backups will have a failure status and will send alerts, so you will not forget about the need to create new scripts.
For Windows instances with a backup agent configured:
If that was the configuration of the original policy, the new instance (assuming it is a Windows instance) will also be assigned as the policy agent. However, since it does not have an authentication key, and since the agent needs to be installed and configured on the instance, the backups will have a failure status. Setting the new authentication key and installing the agent needs to be done manually.
Auto Target Removal for the new policy will always be set to yes and alert, regardless of the setting of the template policy. The basic assumption is that a policy created by a tag will automatically remove resources which do not exist anymore, which is the equivalent as if their tag was deleted.
13.1.3 – Setting Backup Options for EC2 Instances
When adding an instance to a policy, or creating a new policy from template, you may make a few decisions about the instance:
- To create snapshots only for this instance.
- To create snapshots with an initial AMI.
- To schedule AMI creation only.
If this option is not set, CPM will assume the default:
- Snapshots only for Linux.
- Snapshots with initial AMI for Windows instances by adding a backup option after the policy name. The backup option can be one of the following values:
For example, with existing policy: policy1#only-snaps.
Or, for a new policy based on template and setting AMI creation: my_new_policy:existing_policy#only-amis
Note: The only-amis option will create AMIs without rebooting them. The option only-amis-reboot will create AMIs with reboot.
For a Windows instance, you can also define backup with app-aware, i.e. a backup agent. It is used the same as the snapshots and AMI options.
- When adding the app-aware option, the agent is set to the default: VSS is enabled and backup scripts are disabled.
- Additional configurations need to be done manually, and not with the tag.
You can also combine the backup options: policy1#initial-ami#app-aware.
13.1.4 – Tagging a Resource to be Removed from All Policies
By creating the cpm backup tag with the value no-backup (lower case), you can tell CPM to remove this resource from all policies.
Tag scanning can only be controlled by the admin/root user. When scanning is running, it will do so for all the users in the system but will only scan AWS accounts that have Scan Resources enabled. This setting is disabled by default. CPM will automatically scan resources in all AWS regions.
In the General Settings screen you can enable or disable tag scanning, and you can set the interval in hours for automatic scans. You also have the Scan Now button to initiate a tag scan immediately.
Note: Even if scanning is disabled, clicking Scan Now will initiate a scan.
If you do want automated scans to run, keep scanning enabled and set the interval in hours between scans using the General Settings screen. You will also need to set Scan Resources for the relevant AWS accounts.
13.3.1 – Pitfalls
There are potential issues you should try to avoid when managing your backup via tags:
- The first is not to create contradictions between the tags content and manual configuration. If you tag a resource and it is added to a policy, and later you remove it from the policy manually, it may come back at the next tag scan. CPM tries to warn you from such mistakes.
- Policy name changes can also affect tag scanning. If you rename a policy, the policy name in the tag can be wrong. When renaming a policy, correct any relevant tag values.
- When you open a policy that was created by a tag scan to edit it, you will see a message at the top of the dialog window: “* This policy was automatically added by tag scan”.
Note: Even if all the backup targets are removed, CPM will not delete any policy on its own, since deletion of a policy will also delete all its data. If you have a daily summary configured (see section 15.5), policies without backup targets will be listed.
- If the same AWS account is added as multiple accounts in CPM, the same tags can be scanned multiple times, and the behavior can become unpredictable. N2WS generally discourages this practice. It is better to define an account once, and then allow delegates (see section 16.4) access to it. If you added the same AWS account multiple times (even for different users), make sure only one of the accounts in CPM has Scan Resources enabled.
13.3.2 – Troubleshooting
Sometimes you need to understand what happened during a tag scan, especially if the tag scan did not behave as expected, such as a policy was not created. In the General Settings screen, you can view the log of the last tag scan and see what happened during this scan, as well as any other problems (e.g. problem parsing the tag value) that were encountered. Also, if the daily summary is enabled, new scan results from the last day will be listed in the summary.
Ensure tag format is correct
Tips for ensuring correct tag formats are:
- When listing multiple policy names, make sure they are separated by spaces.
- When creating new policy, verify using a colon ‘:’ and not a semi-colon ‘;’. The syntax is new_policy1:existing_policy1.
- Use a valid name for the new policy or it will not be created. An error message will be added to scan log.
- Use correct names for existing/template policies.
- Resource scanning order is NOT defined, so use policy names as existing/template only if you are sure that it exists in CPM – defined manually or scanned previously.