Using Bitbucket Server DIY Backup in AWS
This page describes how to execute a DIY Backup and Restore of a Bitbucket Server instance deployed in AWS.
About the Bitbucket Server DIY Backup for AWS
Bitbucket Server DIY Backup for AWS leverages AWS infrastructure to backup and restore the application. The provided scripts take EBS snapshots of the volume containing the Bitbucket Server shared home directory and the database data directory. These snapshots can later be used to create a new volume and attach it to your instance thus restoring Bitbucket Server to a specific point in time.
Other approaches include using the Bitbucket Server Backup Client or manually running the same steps as the Bitbucket Server DIY Backup for AWS yourself. The benefits of using the Bitbucket Server DIY Backup over these approaches are:
- taking AWS native snapshots are faster than filesystem level copying
- downtime is kept to an absolute minimum
- snapshots are stored with the redundancy and durability of S3
- it makes it easy to relocate an instance to a different Availability Zone in the future
The scripts use the AWS CLI toolset, which is included in all instances launched from the AMI, regardless of launch method. The template creates an IAM role with a policy that grants the instance the permissions required to backup and restore the EBS volume. See the Advanced configuration section below for an example policy with similar permissions.
Configure the Bitbucket Server DIY Backup script
The provided script, bitbucket.diy-aws-backup.vars.sh
, comes with sensible defaults for your AWS environment, but you need to modify variables in the script for your specific setup. These variables indicate how to lock Bitbucket Server for backup, to ensure consistency by preventing writes to the volume.
To locate the Bitbucket Server DIY Backup scripts available in the default installation directory:
Connect to your Bitbucket Server instance on AWS over SSH, use
ec2-user
as the user name, for example:ssh -i keyfile.pem ec2-user@ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com
Go to the
bitbucket-diy-backup
directory.
cd /opt/atlassian/bitbucket-diy-backup
Pull the latest version of the scripts. The installation directory contains a clone of the Bitbucket Server DIY Backup scripts repository. You can update to the latest changes at any time.
git pull
Locate the variables script.
bitbucket.diy-aws-backup.vars.sh
Modify the variables within the script for your specific instance.
Variables
These variables must be customized.
Variable | Explanation |
---|---|
BITBUCKET_BACKUP_USER | The username of a Bitbucket Server user with the SYSADMIN role. |
BITBUCKET_BACKUP_PASS | The password to the |
Users who have attached their home directory volume to a device name other than /dev/xvdf
may also need to update the HOME_DIRECTORY_DEVICE_NAME variable. See the Advanced configuration section for more information.
Back up your instance
To back up your Bitbucket Server instance within AWS you need to run the bitbucket.diy-aws-backup.sh
script. This will take the values you configured above to perform the backup.
To run the Bitbucket Server DIY Backup scripts
Connect to your Bitbucket Server instance on AWS over SSH, use
ec2-user
as the user name, for example:ssh -i keyfile.pem ec2-user@ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com
Go to the
bitbucket-diy-backup
directory.cd /opt/atlassian/bitbucket-diy-backup
Run the
bitbucket.diy-aws-backup.sh
../bitbucket.diy-aws-backup.sh
Running the scripts will produce an output similar to this:
You can review the newly created snapshot in the AWS EC2 console.
Restore your instance
Restoring your instance is done by replacing the existing volume with a new one created using an existing EBS snapshot.
To restore your instance using the Bitbucket Server DIY Restore scripts
Connect to your Bitbucket Server instance on AWS over SSH, use
ec2-user
as the user name, for example:ssh -i keyfile.pem ec2-user@ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com
Go to the
bitbucket-diy-backup
directory.cd /opt/atlassian/bitbucket-diy-backup
Stop your Bitbucket Server instance.
sudo /etc/init.d/atlbitbucket stop
Stop your database (if available).
sudo /etc/init.d/postgresql93 stop
Unmount any volumes using the configured mount point.
sudo umount /media/atl
Detach any volumes using the
/dev/sdf
device name. These commands export the instance region for AWS CLI tools, retrieve the ID for the volume attached to the device name, and detach the volume from the instance.export AWS_DEFAULT_REGION=`curl -s -f http://169.254.169.254/latest/meta-data/placement/availability-zone | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'` DETACH_VOLUME_ID=$(aws ec2 describe-volumes --filter Name=attachment.instance-id,Values=`curl -s -f http://169.254.169.254/latest/meta-data/instance-id` Name=attachment.device,Values=/dev/xvdf | jq -r '.Volumes[0].VolumeId') aws ec2 detach-volume --volume-id ${DETACH_VOLUME_ID}
The previous commands assume the home directory volume is attached to device name
/dev/xvdf
All snapshot taken using the DIY backup script (see above) will be tagged with key "Name" and value which includes the configured
INSTANCE_NAME
and a timestamp for when the backup was taken. To see which snapshots are available for yourINSTANCE_NAME
, run the script without any arguments../bitbucket.diy-aws-restore.sh
The snapshots are sorted alphabetically to keep the latest snapshots at the top of the list.
After you select a tag, run the script again with the instance tag as an argument.
./bitbucket.diy-aws-restore.sh bitbucket-20150326-013036-405
You can review the newly created volume in the AWS EC2 console.
Start your instance.
sudo /etc/init.d/postgresql93 start sudo /etc/init.d/atlbitbucket start
Advanced configuration
Variable | Explanation |
---|---|
INSTANCE_NAME | A name for your instance. It cannot contain spaces. It must be under 100 characters in length. This will be used as a prefix when tagging your instance snapshot. |
BITBUCKET_URL | The Bitbucket Server base URL. It must not end with '/' |
BITBUCKET_HOME | The path to the Bitbucket Server home directory. For example /var/atlassian/application-data/bitbucket |
BACKUP_DATABASE_TYPE | ' |
BACKUP_HOME_TYPE | ' |
HOME_DIRECTORY_MOUNT_POINT | The mount point for the volume holding the BITBUCKET_HOME directory as it appears in |
HOME_DIRECTORY_DEVICE_NAME | The device name on to which the BITBUCKET_HOME volume is attached as it appears in the Amazon console (for example /dev/sdf ) |
AWS_AVAILABILITY_ZONE | The availability zone for your AWS instance. If left unchanged from the template it will be retrieved from the metadata endpoint |
AWS_REGION | The region for your AWS instance. If left unchanged from the template it will be derived from the |
RESTORE_HOME_DIRECTORY_VOLUME_TYPE | The type of volume to create from the snapshot (one of io1, gp2, and standard) |
RESTORE_HOME_DIRECTORY_IOPS | The provisioned IOPS for the new volume. Only necessary if |
HIPCHAT_URL | The url for the HipChat API |
HIPCHAT_ROOM | The HipChat room to which notifications should be delivered |
HIPCHAT_TOKEN | The authentication token for the HipChat API |
CURL_OPTIONS | A set of options to pass to the curl commands executed by the scripts |
BITBUCKET_VERBOSE_BACKUP | If FALSE info and print level logging will be skipped |
Setting up the instance role
The DIY backup and restore scripts use the AWS CLI toolset to do their job. These tools need to authenticate with AWS in order to gain access to your resources (EBS volumes, snapshots, etc). The recommended way of providing credentials to the instance is by launching it with an instance role that has a suitable policy attached. If you are using the Bitbucket Server CloudFormation template, it will take care of creating a policy for you and attach it to the instance at launch time.
If you need to create your own policy, you can use this JSON object as an example of the minimum permissions required for an instance:
{
"Statement": [
{
"Resource": [
"*"
],
"Action": [
"ec2:AttachVolume",
"ec2:CreateSnapshot",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DescribeSnapshots",
"ec2:DescribeVolumes",
"ec2:DetachVolume"
],
"Effect": "Allow"
}
],
"Version": "2012-10-17"
}
For other ways of configuring the AWS CLI toolset, please refer to the documentation.