15. Alerts, Notifications and Reporting

Contents

 

 

15 – Alerts, Notifications and Reporting

CPM manages the backup operations of your EC2 servers. In order to notify you when something is wrong and to integrate with your other cloud operations, CPM allows sending alerts, notifications and even raw reporting data. So, if you have a network operations center (NOC), are using external monitoring tools or just want an email to be sent to the system administrator whenever a failure occurs, CPM has an answer for that.

 

15.1 – Alerts

Alerts are notifications about issues in your CPM backup solution. Whenever a policy fails, in backup or DR, an alert is issued so you will know this policy is not functioning properly. Later, when the policy succeeds, the alert is turned off or deleted, so you will know that the issue is resolved. Alerts can be issued for failures in backup and DR, as well as general system issues like license expiration (for relevant installations).

 

 

15.2 – Pull Alerts

If you wish to integrate CPM with 3rd party monitoring solutions, CPM allows API access to pull alerts out of CPM. A monitoring solution can call this API to check if CPM has alerts. When calling this API, the caller receives the current alerts in JSON format. The call is an HTTPS call, and if you configured CPM server to use an alternate port (not 443), you will need to use that port for this API call as well. CPM requires an authentication key from the caller. Every CPM user can define such a key to get the relevant alerts. The root user can also get relevant alerts from other managed users, but not from independent users.

15. Alerts, Notifications and Reporting

Figure 15‑1

 

To configure an API call:

  1. At the top of any screen, click the Notifications button.
  2. In the notifications screen, click the Configure API Authentication Key link.
  3. In the popup screen, select Enable or Disable in the API Access list.
  4. To generate an authentication key, click new authentication key (see Figure 15‑1).
  5. To overwrite any key in the Authentication Key box, click new authentication key.
  6. After enabling and setting the key, you can use the API call to get all alerts:

https://<your CPM Server address>:<your port>/agentapi/get_cpm_alerts/

 

A simple example of Python is:

d:\tmp>python

Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32

Type "help", "copyright", "credits" or "license" for more information.

>>> import urllib2, json

>>> server_address = 'ec2-54-228-126-14.compute-1.amazonaws.com'

>>> server_port = 443

>>> authkey = 'afb488681baf0132fe190315e87731f883a7dac548c08cf58ba0baddc7006132a

a74f99ab07eff736477dca86b460a4b1a7bfe826e16fdbc'

>>> url = 'https://%s:%d/agentapi/get_cpm_alerts/' % (server_address, server_port)

>>> url

'https://ec2-54-228-126-14.compute-1.amazonaws.com:443/agentapi/get_cpm_alerts/'

>>> request = urllib2.Request (url)

>>> request.add_header("Authorization", authkey)

>>> handle = urllib2.urlopen (request)

>>> answer = json.load (handle)

>>> handle.close ()

>>> answer

[{u'category': u'Backup', u'message_body': u'Policy win_server (user: root, account: main) -
backup that started at 07/20/2013 09:00:00 AM failed. Last successful backup was at
07/20/2013 08:00:00 AM', u'severity': u'E', u'title': u'Policy win_server Backup Failure',
u'alert_time': u'2013-07-20 06:00:03', u'policy': {u'name': u'win_server'}}, {u'category': u'Backup',
u'message_body': u'Policy web_servers (user: root, account: main) - backup that started at
07/20/2013 09:20:03 AM failed. Last successful backup was at 07/20/2013 08:30:00 AM',
u'severity':u'E', u'title': u'Policy web_servers Backup Failure', u'alert_time': u'2013-07-20 06:22:12',
u'policy': {u'name': u'web_servers'}}]

>>>

 

The JSON response is a list of alert objects, each containing the following fields:

  • category
  • title
  • message_body
  • alert_time (time of the last failure)
  • policy
  • severity

 

 

15.3 – Using SNS

CPM can also push alerts to notify you of any malfunction or issue via SNS. To use it, your account needs to have SNS enabled. SNS can send push requests via email, HTTP/S, SQS, and depending on location, SMS.

 

With SNS you create a topic, and for each topic there can be multiple subscribers and multiple protocols. Every time a notification is published to a topic, all subscribers get notified. For more information about SNS, see https://aws.amazon.com/sns/.

 

CPM can create the SNS topic for you and subscribe the user email defined in the configuration phase. To add subscribers, go to the SNS Dashboard in the AWS Management console), add a recipient, and choose a protocol (SMS, HTTP, etc.), A link to this console is in the CPM’s notifications screen.

 

For the small volume of SNS messages CPM uses, there is usually no cost or it is negligible. For SNS pricing see https://aws.amazon.com/sns/pricing/.

 

15.3.1 – Configuring SNS

To configure CPM for SNS, click the Notifications button at the top of any screen.

15. Alerts, Notifications and Reporting

Figure 15‑2

The Notifications screen appears as shown in Figure 15‑2.

 

To use SNS:

  • You will need to enter AWS account credentials for the SNS service.
  • There is one notifications configuration per user, but there can be multiple AWS accounts (where applicable).
  • SNS credentials are not tied to any of the backed-up AWS accounts. You can choose a region, and enter credentials, which can be regular credentials, IAM user (see section 14.3). To use the CPM Server instance’s IAM role (only for the root user), type use_iam_role for both access and secret keys.
  • If you are the root (main) user, you can also choose whether to include or exclude alerts about managed users (see section 16.2).
  • SNS is used both for push alerts and for sending a daily summary.

 

 

15.4 – Push Alerts

Push alerts use SNS to send notifications about malfunctions and issues in CPM’s operation.

 

To enable push alerts:

  1. Set Alerts to Enabled.
  2. Either paste in the topic’s ARN that you copied from the SNS tab of the AWS Management Console, or request CPM to create the topic for you and add the user’s email as a recipient (optional).

Each recipient will receive a message requesting subscription confirmation before receiving alerts.

 

 

15.5 – Daily Summary

Daily summary is a message that is sent once a day, summarizing all current alerts in the system. It can be configured instead of, or in addition to, regular alerts. It can be useful for several reasons:

  • If you are experiencing issues frequently it sometimes reduces noise to get a daily summary. Furthermore, since backup is the second line of defense, some people feel they do not need to get an instant message on every backup issue that occurs.
  • Even if there are no issues, a daily summary is a reminder that all is ok. If something happens and CPM crashed altogether, and your monitoring solution did not report it, you will notice daily summaries will stop.
  • The Daily summary contains a list of policies which are disabled and policies that do not have schedules assigned to them. Although neither is an error, sometimes someone can accidentally leave a policy disabled or without a schedule and not realize that it is not working.

C:\Users\Janet\Documents\ScreenHunter\ScreenHunter_1296 Mar. 22 15.31.jpg

Figure 15‑3

While configuring SNS, as shown in Figure 15‑3, you can also configure the Daily Summary.

 

To configure the Daily Summary:

1. In the Notification Settings screen, select Enabled in the Daily Summary list.

2. Use one of following options for defining the Daily Summary topic:

  • If you have Alerts configured and you want to use the same SNS topic for summaries, select the same topic as alerts check box.
  • To create a new topic, select the Create new topic check box, and complete the next screen.
  • Type or paste an ARN in the Summary Topic box.

There is an advantage of using a separate topic since sometimes you want different recipients: It makes sense for a system admin to get alerts by SNS, but to get the daily summary by email only. The display name of the topic also appears in the message (in emails it appears as the sender name), so with separate topics it is easier to distinguish alerts.

3. To add a recipient, select the Add user email as recipient check box, and complete the next screen.

4. In the Send Daily Summary at lists, select the hour and minutes to send the notification.

 

15.6 – Raw Reporting Data

In the future, N2WS plans on adding a full-scale reporting module to CPM. In the meantime, you can get two raw reports that you can download in CSV format (Comma Separated Values). These reports are for the logged-in user. For the root user, they will include also data of other managed users. These reports include all the records in the database; you can filter or create graphic reports from them by loading them to a spreadsheet or reporting tool. The two reports combined give a complete picture of backups and snapshots taken by CPM.

 

To download the CSV reports, click the backup view report or snapshot view report link at the bottom of CPM’s main screen.

 

15.6.1 – Backup View CSV Report

This report will have a record for each backup (similar to the backup monitor) with details for each of the backups:

  • Backup ID – A unique numerical ID representing the backup.
  • User – Name of the User if the system has multiple users and the user downloading the report is root.
  • Account – Name of the AWS account.
  • Policy – Name of the policy.
  • Status – Status of the backup, same is in the backup monitor.
  • DR Status – Status of DR, same as in the backup monitor.
  • Start Time – Time the backup started.
  • End Time – Time the backup ended.
  • Is RetryYes if this backup was a retry after failure, otherwise no.
  • Marked for DeletionYes if this backup was marked for deletion. If yes, the backup no longer appears in the backup monitor and is not recoverable.

 

15.6.2 – Snapshot View CSV Report

This report will have a record for each EBS or RDS snapshot in the database:

  • Backup ID – ID of the backup the snapshot belongs to. Matches the same snapshots in the previous report.
  • Account – Name of the AWS account.
  • Region – AWS region.
  • Type – Type of snapshot: EBS, RDS or EBS Copy, which is a DR copied snapshot.
  • Volume/DB – AWS ID of the backed up EBS volume or RDS database.
  • Instance – If this snapshot belongs to a backed up EC2 instance, the value will be the AWS ID of that instance, otherwise it will contain the string: None.
  • Snapshot ID – AWS ID of the snapshot.
  • SucceededYes or No.
  • Start Time – Time the snapshot started.
  • End Time – Time the snapshot ended.
  • Deleted At – Time of deletion, or N/A, if the snapshot was not deleted yet.

 

15.6.3 – Keeping Records after Deletion

By default, when a backup is marked for deletion, it will be deleted right away from the CPM database, and therefore not appear in the reports. There are exceptions, such as if CPM could not delete all the snapshots in a backup (e.g. a snapshot is included in an AMI and cannot be deleted). Sometimes you need to save records for a period of time after they were marked for deletion for compliance, such as General Certificate of Conformity (GCC).

 

To save records for a period of time:

1. On the CPM server, create a file containing only the number of days to save records in /cpmdata/conf/num_days_to_keep_backup_records.

2. Set the file permissions to allow all users read access.

Note: The number of days is counted since the backup was created and not deleted. If you want to make sure every backup record is saved for 90 days after creation, even if it was already deleted, you need to put 90 in the file.

3. A typical way to create it is as follows:

echo 90 > /cpmdata/conf/num_days_to_keep_backup_records

 

To see how to login to the CPM Server instance using SSH, see section 7.1.

Note: Keeping backups for long periods of time can cause the CPM database to grow and therefore affect the size you need to allocate for CPM’s data volume. N2WS estimates that every GiB will accommodate the backup of 10 instances. N2WS estimates that 10 instances are correct when every record is kept for around 30 days. If you want to keep records for 90 days, triple the estimate, i.e. for 10 instances make the volume 3 GiB, for 20 make the volume 6 GiB, etc.

 

 

15.7 – Usage Reports

In addition to the raw reports, you can also download CSV usage reports. A usage report for a user will give the number of AWS accounts, instance and non-instance storage this user is consuming. This can be helpful for inter-user accounting.

  • For each user, there is a link usage report for current user.
  • For the root user, there is also a link usage report for all users which will give all the breakdown of usage between all the users on the CPM server.

 

 

15.8 – Protected and Unprotected Resources Reports

The protected and unprotected resources reports provide information about the AWS resources with and without backup protection. Both reports are available in the Accounts management screen. At the bottom of the main screen, the unprotected resources report (all users) is also available for admin/root users and the unprotected resources report (current user) for other users.

 

15.8.1 – Protected Resources

The protected resources report contains information about the AWS resources with backup policies.

  • User Name (on all users reports)
  • ID for the resource
  • AWS resource name
  • Region
  • Polices
  • Schedules

 

The protected resources report is available immediately for the current user or all users depending on the account type.

 

15.8.2 – Unprotected Resources

The unprotected resources report contains information about the AWS resources that do not have backup policies.

  • Resource Type
  • Name of resource
  • Resource ID
  • Region
  • Partial
  • Account
  • User
  • Count of number of unprotected resources per resource type.

 

To create the unprotected resources report:

  1. Click the unprotected resources report (current user/all users) link in the Reports column of the Accounts management screen or at the bottom of the main screen.
  2. In the Notifications management screen, click the View Alerts to check if the report has completed.
  3. If completed, click the download last unprotected resource report link at the bottom of the main screen.
  4. Check for the report in your Downloads folder.

 

15.9 – Reports Page

With version 2.3.0, there is a new Reports button which contains download links for the following reports:

  • Backups – Filter for Account and From/To Date/Time
  • Snapshots – Filter for Account and From/To Date/Time
  • Audit – Filter for User and From/To Date/Time
  • Usage – Filter for User
  • Protected Resources Report – Filter for User and Account

 

To generate a report:

1. Under the target report, select the relevant filter value in the Account or User drop-down list.

15. Alerts, Notifications and Reporting

Figure 15‑4

 

2. To filter by date and time, click the calendar icons and select the From and To date and time values.

15. Alerts, Notifications and Reporting

Figure 15‑5

3. Click the Download link to generate and download the report.