Using Stash DIY Backup in AWS

This page describes how to execute a DIY Backup and Restore of a Stash instance deployed in AWS.

About the Stash DIY Backup for AWS

Stash DIY Backup for AWS leverages AWS infrastructure to backup and restore Stash. The provided scripts take EBS snapshots of the volume containing the Stash shared home directory and the database data directory. These snapshots can later be used to create a new volume and attach it to your instance thus restoring Stash to a specific point in time.

Other approaches include using the Stash Backup Client or manually running the same steps as the Stash DIY Backup for AWS yourself. The benefits of using the Stash DIY Backup over these approaches are:

taking AWS native snapshots are faster than filesystem level copying
downtime is kept to an absolute minimum
snapshots are stored with the redundancy and durability of S3
it makes it easy to relocate an instance to a different Availability Zone in the future

The scripts use the AWS CLI toolset, which is included in all instances launched from the AMI, regardless of launch method. The template creates an IAM role with a policy that grants the instance the permissions required to backup and restore the EBS volume. See the Advanced configuration section for an example policy with similar permissions.

On this page

Configure the Stash DIY Backup script

The provided script, stash.diy-aws-backup.vars.sh, comes with sensible defaults for your AWS environment, but you need to modify variables in the script for your specific setup. These variables indicate how to lock Stash for backup, to ensure consistency by preventing writes to the volume.

To locate the Stash DIY Backup scripts available in the default installation directory:

Connect to your Stash instance on AWS over SSH, use ec2-user as the user name, for example:
```
 ssh -i keyfile.pem ec2-user@ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com
```
Go to the stash-diy-backup directory.
```
cd /opt/atlassian/stash-diy-backup
```
Pull the latest version of the scripts. The installation directory contains a clone of the Stash DIY Backup scripts repository. You can update to the latest changes at any time.
```
git pull
```
Locate the variables script.
```
stash.diy-aws-backup.vars.sh
```
Modify the variables within the script for your specific instance.

Variables

These variables must be customized.

Variable	Explanation
STASH_BACKUP_USER	The username of a Stash user with the SYSADMIN role.
STASH_BACKUP_PASS	The password to the `STASH_BACKUP_USER` account.

Users who have attached their home directory volume to a device name other than /dev/xvdf may also need to update the HOME_DIRECTORY_DEVICE_NAME variable. See the Advanced configuration section for more information.

Back up your instance

To back up your Stash instance within AWS you need to run the stash.diy-aws-backup.sh script. This will take the values you configured above to perform the backup.

To run the Stash DIY Backup scripts

Connect to your Stash instance on AWS over SSH, use ec2-user as the user name, for example:
```
 ssh -i keyfile.pem ec2-user@ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com
```
Go to the stash-diy-backup directory.
```
cd /opt/atlassian/stash-diy-backup
```
Run the stash.diy-aws-backup.sh.
```
./stash.diy-aws-backup.sh
```
Running the scripts will produce an output similar to this:

[http://localhost:7990]  INFO: Using IAM instance role Stash-DIY-Backup-StashBackupRole-1GJ4BL2XHPDBG
[http://localhost:7990]  INFO: Looking up volume for device name /dev/sdf
[http://localhost:7990]  SUCC: Found volume vol-aae66abd for device name /dev/sdf
[http://localhost:7990]  INFO: locked with '603c1d2c41121d1b6f42c0b03d4e03ee8a22577b'
[http://localhost:7990]  INFO: backup started with 'a62d9b002747877e57d7ff32cdaedc92ba66db79'
[http://localhost:7990]  INFO: Waiting for DRAINED state.. done
[http://localhost:7990]  INFO: db state 'DRAINED'
[http://localhost:7990]  INFO: scm state 'DRAINED'
[http://localhost:7990]  INFO: Backup progress updated to 50
[http://localhost:7990]  INFO: Freezing filesystem at mount point /media/atl
[http://localhost:7990]  INFO: Performing backup of home directory
[http://localhost:7990]  SUCC: Taken snapshot snap-2f133304 of volume vol-aae66abd
[http://localhost:7990]  INFO: Tagged snap-2f133304 with Name=stash-20150326-013036-405
[http://localhost:7990]  INFO: Unfreezing filesystem at mount point /media/atl
[http://localhost:7990]  INFO: Backup progress updated to 100
[http://localhost:7990]  INFO: Stash instance unlocked
[http://localhost:7990]  SUCC: Successfully completed the backup of your Stash instance
[http://localhost:7990]  INFO: Cleaning up...
[http://localhost:7990]  INFO: Unfreezing filesystem at mount point /media/atl

You can review the newly created snapshot in the AWS EC2 console.

Restore your instance

Restoring your instance is done by replacing the existing volume with a new one created using an existing EBS snapshot.

To restore your Stash instance using the Stash DIY Restore scripts

Connect to your Stash instance on AWS over SSH, use ec2-user as the user name, for example:
```
 ssh -i keyfile.pem ec2-user@ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com
```
Go to the stash-diy-backup directory.
```
cd /opt/atlassian/stash-diy-backup
```
Stop your Stash instance.
```
sudo /etc/init.d/atlstash stop
```
Stop your database (if available).
```
sudo /etc/init.d/postgresql93 stop
```
Unmount any volumes using the configured mount point.
```
sudo umount /media/atl
```

Detach any volumes using the /dev/sdf device name. These commands export the instance region for AWS CLI tools, retrieve the ID for the volume attached to the device name, and detach the volume from the instance.

export AWS_DEFAULT_REGION=`curl -s -f http://169.254.169.254/latest/meta-data/placement/availability-zone | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`
DETACH_VOLUME_ID=$(aws ec2 describe-volumes --filter Name=attachment.instance-id,Values=`curl -s -f http://169.254.169.254/latest/meta-data/instance-id` Name=attachment.device,Values=/dev/xvdf | jq -r '.Volumes[0].VolumeId')
aws ec2 detach-volume --volume-id ${DETACH_VOLUME_ID}

The previous commands assume the home directory volume is attached to device name /dev/xvdf

All snapshot taken using the DIY backup script (see above) will be tagged with key "Name" and value which includes the configured INSTANCE_NAME and a timestamp for when the backup was taken. To see which snapshots are available for your INSTANCE_NAME, run the script without any arguments.

./stash.diy-aws-restore.sh

The snapshots are sorted alphabetically to keep the latest snapshots at the top of the list.

[http://localhost:7990]  INFO: Using IAM instance role Stash-DIY-Backup-StashBackupRole-1GJ4BL2XHPDBG
[http://localhost:7990]  INFO: Usage: ./stash.diy-aws-restore.sh <snapshot-tag>
Available snapshot tags:
stash-20150327-013036-405
stash-20150326-234633-252
stash-20150326-231048-106
stash-20150326-231038-587
stash-20150326-224449-846
stash-20150326-224012-288
stash-20150326-222700-117
stash-20150326-222522-962
stash-20150326-220256-274
stash-20150326-083409-384

After you select a tag, run the script again with the instance tag as an argument.

./stash.diy-aws-restore.sh stash-20150326-013036-405

[http://localhost:7990]  INFO: Using IAM instance role Stash-DIY-Backup-StashBackupRole-1GJ4BL2XHPDBG
[http://localhost:7990]  INFO: Restoring from tag stash-20150326-013036-405
[http://localhost:7990]  INFO: Checking for existing volumes using device name /dev/sdf
[http://localhost:7990]  INFO: Found EBS snapshot snap-2f133304 for tag stash-20150326-013036-405
[http://localhost:7990]  INFO: Restoring home directory from snapshot snap-2f133304 into a gp2 volume
[http://localhost:7990]  SUCC: Restored snapshot snap-2f133304 into volume vol-dae16dcd
[http://localhost:7990]  INFO: Waiting for volume vol-dae16dcd to be attached. This could take some time
[http://localhost:7990]  INFO: Volume vol-dae16dcd state: attaching
[http://localhost:7990]  INFO: Volume vol-dae16dcd state: attached
[http://localhost:7990]  SUCC: Attached volume vol-dae16dcd to device /dev/sdf at instance i-6acb8a8e
[http://localhost:7990]  SUCC: Mounted device /dev/sdf to /media/atl
[http://localhost:7990]  INFO: Performed restore of home directory snapshot
[http://localhost:7990]  SUCC: Successfully completed the restore of your Stash instance

You can review the newly created volume in the AWS EC2 console.

Start your instance.

sudo /etc/init.d/postgresql93 start
sudo /etc/init.d/atlstash start

Advanced configuration

Variable	Explanation
INSTANCE_NAME	A name for your instance. It cannot contain spaces. It must be under 100 characters in length. This will be used as a prefix when tagging your instance snapshot.
STASH_URL	The Stash base URL. It must not end with '/'
STASH_HOME	The path to the Stash home directory. For example `/var/atlassian/application-data/stash`
BACKUP_DATABASE_TYPE	'`ebs-collocated`' means the database data directory is located in the same volume as the STASH_HOME.
BACKUP_HOME_TYPE	'`ebs-home`' means the Stash home (and the database data directory if ebs-collocated has been specified) will be backed up by taking a snapshot of an EBS volume.
HOME_DIRECTORY_MOUNT_POINT	The mount point for the volume holding the STASH_HOME directory as it appears in `/etc/fstab` (for example `/media/atl`)
HOME_DIRECTORY_DEVICE_NAME	The device name on to which the STASH_HOME volume is attached as it appears in the Amazon console (for example `/dev/sdf`)
AWS_AVAILABILITY_ZONE	The availability zone for your AWS instance. If left unchanged from the template it will be retrieved from the metadata endpoint
AWS_REGION	The region for your AWS instance. If left unchanged from the template it will be derived from the `AWS_AVAILABILITY_ZONE`
RESTORE_HOME_DIRECTORY_VOLUME_TYPE	The type of volume to create from the snapshot (one of io1, gp2, and standard)
RESTORE_HOME_DIRECTORY_IOPS	The provisioned IOPS for the new volume. Only necessary if `RESTORE_HOME_DIRECTORY_VOLUME_TYPE` is '`io1`'
HIPCHAT_URL	The url for the HipChat API
HIPCHAT_ROOM	The HipChat room to which notifications should be delivered
HIPCHAT_TOKEN	The authentication token for the HipChat API
CURL_OPTIONS	A set of options to pass to the `curl` commands executed by the scripts
STASH_VERBOSE_BACKUP	If `FALSE` `info` and `print` level logging will be skipped

Setting up the instance role

The DIY backup and restore scripts use the AWS CLI toolset to do their job. These tools need to authenticate with AWS in order to gain access to your resources (EBS volumes, snapshots, etc). The recommended way of providing credentials to the instance is by launching it with an instance role that has a suitable policy attached. If you are using the Stash CloudFormation template, it will take care of creating a policy for you and attach it to the instance at launch time.

If you need to create your own policy, you can use this JSON object as an example of the minimum permissions required for an instance:

{
    "Statement": [
        {
            "Resource": [
                "*"
            ],
            "Action": [
                "ec2:AttachVolume",
                "ec2:CreateSnapshot",
                "ec2:CreateTags",
                "ec2:CreateVolume",
                "ec2:DescribeSnapshots",
                "ec2:DescribeVolumes",
				"ec2:DetachVolume"
            ],
            "Effect": "Allow"
        }
    ],
    "Version": "2012-10-17"
}

For other ways of configuring the AWS CLI toolset, please refer to the documentation.

Page tree