07. Linux/Unix Instances Backup

Contents

 

7 – Linux/Unix Instances Backup

Making application-consistent backup of Linux instances does not require any agent installation. Since the CPM server is Linux based, backup scripts will run on it. Typically, such a script would use SSH to connect to the backed-up instance and perform application quiescence. However, this can also be done using custom client software.

 

There is no parallel to VSS in Linux, so the only method available is by running backup scripts.

 

 

7.1 – Connecting to the CPM Server

In order to create, test, and install backup scripts, you will need to connect to the CPM server using SSH with cpmuser. The only way to authenticate cpmuser is by using the private key from the key pair you used when you launched the CPM server instance. As long as your key is not compromised, no unauthorized person will be able to connect to the CPM server.

 

With cpmuser, you will be able to copy (using secure copy), create, and test your scripts. cpmuser is the same user CPM will use to run the scripts. If you need to edit your scripts on the CPM Server, you can use the Vim or nano editors. Nano is simpler to use.

 

7.2 – Backup scripts

Backup scripts should be placed in the path /cpmdata/scripts. If the policy belongs to a CPM user other than the root user, scripts will be located in a subfolder named like the user (e.g. /cpmdata/scripts/cpm_user1). This path resides on the data volume of CPM, and will remain there even if you terminate the CPM server instance and wish to launch a new one. Backup scripts will remain on the data volume, together with all other configuration data. As cpmuser, you have read, write, and execute permissions in this folder.

  • All scripts should exit with the code 0 when they succeed and 1 (or another non-zero code) when they fail.
  • All scripts have a base name (detailed for each script in the coming sections) and may have any addition after the base name (e.g. before_policy1_v11.5.bash).
  • Scripts can be written in any programming language: shell scripts, Perl, Python, or even binary executables.
  • You only have to make sure the scripts can be executed and have the correct permissions.

Warning: Having more than one script with the same base name can result in unexpected behavior. CPM does not guarantee which script it will run, and even to run the same script every backup.

 

There are three scripts for each policy:

  • Before
  • After
  • Complete

 

7.2.1 – Before Script

The before_<policy name>[_optional_addition].<ext> script runs before backup begins. Typically, this script is used to move applications to backup mode. The before script typically leaves the system in a frozen state for a short time until the snapshots of the policy are fired. One option is to issue a freeze command to a file system like xfs.

 

7.2.2 – After Script

The after_<policy name>[_optional_addition].<ext> script runs after all the snapshots of the policy fire. It runs within a few seconds after the before script. This script releases anything that may have been frozen or locked by the before script. This script accepts the success status of the before script. If the before script succeeded, the argument will be 1. If it failed, crashed, or timed out, the argument will be 0.

Note: This is the opposite of the exit status. Think of this as an argument that is true when the before script succeeds.

 

7.2.3 – Complete Script

The complete_<policy name>[_optional addition].<ext> script runs after all snapshots are completed. Usually, it runs quickly since snapshots are incremental. This script can perform clean-up after the backup is complete and is typically used for transaction logs truncation. The script accepts one argument. If the entire backup was successful and all the previous scripts were successful, it will be 1. If any issues or failures happened, it will be 0. If this argument is 1, truncate logs.

 

7.2.4 – Capturing the Output of Backup Scripts

You can have the output of backup scripts collected and saved in the CPM Server, see sections 4.2.3 and 4.2.5.

7.2.5 – Troubleshooting and Debugging Backup Scripts

You can use the output collected by CPM to debug backup scripts. However, the recommended way is to run them independently of CPM, on the CPM Server machine using SSH. You can then see their outputs and fix what is needed. Once the scripts work correctly, you can start using them with CPM. Assuming these scripts are using SSH, during the first execution you will need to approve the SSH key by answering yes at the command line prompt. If you terminate your CPM Server and start a new one, you will need to run the scripts again from the command line and approve the SSH key.

7.2.6 – Example Backup Scripts

Following is an example of a set of backup scripts that use SSH to connect to another instance and freeze a MySQL Database:

  • The before script will flush and freeze the database.
  • The after script will release it.
  • The complete script will truncate binary logs older than the backup.

Note: These scripts are presented as an example without warranties. Test and make sure scripts work in your environment as expected before using them in your production environment.

 

The scripts are written in bash:

before_MyPolicy.bash

#!/bin/bash

ssh -i /cpmdata/scripts/mysshkey.pem sshuser@ec2-host_name.compute-1.amazonaws.com "mysql -u root –p<MySQL root password>" -e 'flush tables with read lock; flush logs;'"

if [ $? -gt 0 ]; then

echo "Failed running mysql freeze" 1>&2

exit 1

else

echo "mysql freeze succeeded" 1>&2

fi

 

This script connects to another instance using SSH, and then runs a MySQL command. Another approach would be to use a MySQL client on the CPM Server, and then the SSH connection will not be necessary.

 

After that script is executed, the CPM server will start the snapshots, and then call the next script:

after_MyPolicy.bash

#!/bin/bash

if [ $1 -eq 0 ]; then

echo "There was an issue running first script" 1>&2

fi

ssh -i /cpmdata/scripts/mysshkey.pem sshuser@ec2-host_name.compute-1.amazonaws.com "date +'%F %H:%M:%S' > sql_backup_time; mysql -u root -p<MySQL root password> -e 'unlock tables;'"

if [ $? -gt 0 ]; then

echo "Failed running mysql unfreeze" 1>&2

exit 1

else

echo "mysql unfreeze succeeded" 1>&2

fi

 

This script checks the status in the first argument and then does two things:

First, it saves an exact timestamp of the of the current backup of the frozen database to a file,

Then, it releases the lock on the MySQL table.

 

After that, when all snapshots succeed, CPM runs the complete script:

complete_MyPolicy.bash

#!/bin/bash

if [ $1 -eq 1 ]; then

cat /cpmdata/scripts/complete_sql_inner |ssh -i /cpmdata/scripts/mysshkey.pem sshuser@ec2-host_name.compute-1.amazonaws.com "cat > /tmp/complete_ssh; chmod 755 /tmp/complete_ssh; /tmp/complete_ssh"

if [ $? -gt 0 ]; then

echo "Failed running mysql truncate logs" 1>&2

exit 1

else

echo "mysql truncate logs succeeded" 1>&2

fi

else

echo "There was an issue during backup - not truncating logs" 1>&2

fi

It calls an inner script, complete_sql_inner:

butime=`<sql_backup_time`

mysql -u root -p<MySQL root password> -e 'PURGE BINARY LOGS BEFORE "'"$butime"'"'

 

These two scripts purge the binary logs only if the complete script receives 1 as the argument. They purge logs earlier than the time in the timestamp files.

 

7.2.7 – Scripts and SSH Access in a Multi-user Environment

If your CPM configuration requires multiple users, which are separated from each other, you may wish to allow users to access CPM using SSH to create and debug backup scripts:

  • Create additional Linux users in the CPM instance and allowing each user access to their own scripts folder only.
  • cpmuser will need to be able to access and execute the scripts of all users. This can be achieved by assigning the user cpmuser as the group of all user subfolders and scripts. Then, if given executable permissions for the group, cpmuser will be able to access and execute all scripts.