Cloud
Data Center 4.7
Versions

Using Bitbucket Server DIY Backup in AWS

Getting started with Bitbucket Server and AWS

On this page

Related content

Still need help?

The Atlassian Community is here for you.

Ask the community

This page describes how to execute a DIY Backup and Restore of a Bitbucket Server instance deployed in AWS.

About the Bitbucket Server DIY Backup for AWS

Bitbucket Server DIY Backup for AWS leverages AWS infrastructure to backup and restore the application. The provided scripts take EBS snapshots of the volume containing the Bitbucket Server shared home directory and the database data directory. These snapshots can later be used to create a new volume and attach it to your instance thus restoring Bitbucket Server to a specific point in time.

Other approaches include using the Bitbucket Server Backup Client or manually running the same steps as the Bitbucket Server DIY Backup for AWS yourself. The benefits of using the Bitbucket Server DIY Backup over these approaches are:

  • taking AWS native snapshots are faster than filesystem level copying
  • downtime is kept to an absolute minimum
  • snapshots are stored with the redundancy and durability of S3
  • it makes it easy to relocate an instance to a different Availability Zone in the future

The scripts use the AWS CLI toolset, which is included in all instances launched from the AMI, regardless of launch method. The template creates an IAM role with a policy that grants the instance the permissions required to backup and restore the EBS volume. See the Advanced configuration section below for an example policy with similar permissions.

Configure the Bitbucket Server DIY Backup script

The provided script, bitbucket.diy-aws-backup.vars.sh, comes with sensible defaults for your AWS environment, but you need to modify variables in the script for your specific setup. These variables indicate how to lock Bitbucket Server for backup, to ensure consistency by preventing writes to the volume.

To locate the Bitbucket Server DIY Backup scripts available in the default installation directory:

  1. Connect to your Bitbucket Server instance on AWS over SSH, use ec2-user as the user name, for example:

     ssh -i keyfile.pem ec2-user@ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com

  2. Go to the bitbucket-diy-backup directory.

    cd /opt/atlassian/bitbucket-diy-backup

  3. Pull the latest version of the scripts. The installation directory contains a clone of the Bitbucket Server DIY Backup scripts repository. You can update to the latest changes at any time.

    git pull

  4. Locate the variables script.

    bitbucket.diy-aws-backup.vars.sh

  5. Modify the variables within the script for your specific instance. 

Variables

These variables must be customized.

VariableExplanation
BITBUCKET_BACKUP_USER

The username of a Bitbucket Server user with the SYSADMIN role.

BITBUCKET_BACKUP_PASS

The password to the BITBUCKET_BACKUP_USER account.

Users who have attached their home directory volume to a device name other than /dev/xvdf may also need to update the HOME_DIRECTORY_DEVICE_NAME variable. See the Advanced configuration section for more information.

 

Back up your instance

To back up your Bitbucket Server instance within AWS you need to run the bitbucket.diy-aws-backup.sh script. This will take the values you configured above to perform the backup.

To run the Bitbucket Server DIY Backup scripts 

  1. Connect to your Bitbucket Server instance on AWS over SSH, use ec2-user as the user name, for example:

     ssh -i keyfile.pem ec2-user@ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com

  2. Go to the bitbucket-diy-backup directory.

    cd /opt/atlassian/bitbucket-diy-backup

  3. Run the bitbucket.diy-aws-backup.sh.

    ./bitbucket.diy-aws-backup.sh

    Running the scripts will produce an output similar to this:

Click here to see example output... 
[http://localhost:7990]  INFO: Using IAM instance role Bitbucket-DIY-Backup-BitbucketBackupRole-1GJ4BL2XHPDBG
[http://localhost:7990]  INFO: Looking up volume for device name /dev/sdf
[http://localhost:7990]  SUCC: Found volume vol-aae66abd for device name /dev/sdf
[http://localhost:7990]  INFO: locked with '603c1d2c41121d1b6f42c0b03d4e03ee8a22577b'
[http://localhost:7990]  INFO: backup started with 'a62d9b002747877e57d7ff32cdaedc92ba66db79'
[http://localhost:7990]  INFO: Waiting for DRAINED state.. done
[http://localhost:7990]  INFO: db state 'DRAINED'
[http://localhost:7990]  INFO: scm state 'DRAINED'
[http://localhost:7990]  INFO: Backup progress updated to 50
[http://localhost:7990]  INFO: Freezing filesystem at mount point /media/atl
[http://localhost:7990]  INFO: Performing backup of home directory
[http://localhost:7990]  SUCC: Taken snapshot snap-2f133304 of volume vol-aae66abd
[http://localhost:7990]  INFO: Tagged snap-2f133304 with Name=bitbucket-20150326-013036-405
[http://localhost:7990]  INFO: Unfreezing filesystem at mount point /media/atl
[http://localhost:7990]  INFO: Backup progress updated to 100
[http://localhost:7990]  INFO: Bitbucket instance unlocked
[http://localhost:7990]  SUCC: Successfully completed the backup of your Bitbucket instance
[http://localhost:7990]  INFO: Cleaning up...
[http://localhost:7990]  INFO: Unfreezing filesystem at mount point /media/atl

You can review the newly created snapshot in the AWS EC2 console.

Restore your instance

Restoring your instance is done by replacing the existing volume with a new one created using an existing EBS snapshot.

To restore your instance using the Bitbucket Server DIY Restore scripts

  1. Connect to your Bitbucket Server instance on AWS over SSH, use ec2-user as the user name, for example:

     ssh -i keyfile.pem ec2-user@ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com

  2. Go to the bitbucket-diy-backup directory.

    cd /opt/atlassian/bitbucket-diy-backup

  3. Stop your Bitbucket Server instance.

    sudo /etc/init.d/atlbitbucket stop

  4. Stop your database (if available).

    sudo /etc/init.d/postgresql93 stop

  5. Unmount any volumes using the configured mount point.

    sudo umount /media/atl

  6. Detach any volumes using the /dev/sdf device name. These commands export the instance region for AWS CLI tools, retrieve the ID for the volume attached to the device name, and detach the volume from the instance.

    export AWS_DEFAULT_REGION=`curl -s -f http://169.254.169.254/latest/meta-data/placement/availability-zone | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`
DETACH_VOLUME_ID=$(aws ec2 describe-volumes --filter Name=attachment.instance-id,Values=`curl -s -f http://169.254.169.254/latest/meta-data/instance-id` Name=attachment.device,Values=/dev/xvdf | jq -r '.Volumes[0].VolumeId')
aws ec2 detach-volume --volume-id ${DETACH_VOLUME_ID}

    The previous commands assume the home directory volume is attached to device name /dev/xvdf

  7. All snapshot taken using the DIY backup script (see above) will be tagged with key "Name" and value which includes the configured INSTANCE_NAME and a timestamp for when the backup was taken. To see which snapshots are available for your INSTANCE_NAME, run the script without any arguments.

    ./bitbucket.diy-aws-restore.sh

    The snapshots are sorted alphabetically to keep the latest snapshots at the top of the list.

    Click here to see example output... 
    [http://localhost:7990]  INFO: Using IAM instance role Bitbucket-Server-DIY-Backup-Bitbucket-ServerBackupRole-1GJ4BL2XHPDBG
[http://localhost:7990]  INFO: Usage: ./bitbucket.diy-aws-restore.sh <snapshot-tag>
Available snapshot tags:
bitbucket-20150327-013036-405
bitbucket-20150326-234633-252
bitbucket-20150326-231048-106
bitbucket-20150326-231038-587
bitbucket-20150326-224449-846
bitbucket-20150326-224012-288
bitbucket-20150326-222700-117https://confluence.atlassian.com/pages/editpage.action?pageId=776640203#
bitbucket-20150326-222522-962
bitbucket-20150326-220256-274
bitbucket-20150326-083409-384

  8. After you select a tag, run the script again with the instance tag as an argument.

    ./bitbucket.diy-aws-restore.sh bitbucket-20150326-013036-405
    Click here to see example output... 
    [http://localhost:7990]  INFO: Using IAM instance role Bitbucket-Server-DIY-Backup-Bitbucket-ServerBackupRole-1GJ4BL2XHPDBG
[http://localhost:7990]  INFO: Restoring from tag bitbucket-20150326-013036-405
[http://localhost:7990]  INFO: Checking for existing volumes using device name /dev/sdf
[http://localhost:7990]  INFO: Found EBS snapshot snap-2f133304 for tag bitbucket-20150326-013036-405
[http://localhost:7990]  INFO: Restoring home directory from snapshot snap-2f133304 into a gp2 volume
[http://localhost:7990]  SUCC: Restored snapshot snap-2f133304 into volume vol-dae16dcd
[http://localhost:7990]  INFO: Waiting for volume vol-dae16dcd to be attached. This could take some time
[http://localhost:7990]  INFO: Volume vol-dae16dcd state: attaching
[http://localhost:7990]  INFO: Volume vol-dae16dcd state: attached
[http://localhost:7990]  SUCC: Attached volume vol-dae16dcd to device /dev/sdf at instance i-6acb8a8e
[http://localhost:7990]  SUCC: Mounted device /dev/sdf to /media/atl
[http://localhost:7990]  INFO: Performed restore of home directory snapshot
[http://localhost:7990]  SUCC: Successfully completed the restore of your Bitbucket instance

    You can review the newly created volume in the AWS EC2 console.

  9. Start your instance.

    sudo /etc/init.d/postgresql93 start
sudo /etc/init.d/atlbitbucket start

Advanced configuration

VariableExplanation
INSTANCE_NAME
A name for your instance. It cannot contain spaces. It must be under 100 characters in length. This will be used as a prefix when tagging your instance snapshot.
BITBUCKET_URL
The Bitbucket Server base URL. It must not end with '/'
BITBUCKET_HOME
The path to the Bitbucket Server home directory. For example /var/atlassian/application-data/bitbucket
BACKUP_DATABASE_TYPE

'ebs-collocated' means the database data directory is located in the same volume as the BITBUCKET_HOME.

BACKUP_HOME_TYPE

'ebs-home' means the Bitbucket Server home (and the database data directory if ebs-collocated has been specified) will be backed up by taking a snapshot of an EBS volume.

HOME_DIRECTORY_MOUNT_POINT

The mount point for the volume holding the BITBUCKET_HOME directory as it appears in /etc/fstab (for example /media/atl)

HOME_DIRECTORY_DEVICE_NAME
The device name on to which the BITBUCKET_HOME volume is attached as it appears in the Amazon console (for example /dev/sdf)
AWS_AVAILABILITY_ZONE

The availability zone for your AWS instance. If left unchanged from the template it will be retrieved from the metadata endpoint

AWS_REGION

The region for your AWS instance. If left unchanged from the template it will be derived from the AWS_AVAILABILITY_ZONE

RESTORE_HOME_DIRECTORY_VOLUME_TYPE

The type of volume to create from the snapshot (one of io1, gp2, and standard)

RESTORE_HOME_DIRECTORY_IOPS

The provisioned IOPS for the new volume. Only necessary if
RESTORE_HOME_DIRECTORY_VOLUME_TYPE is 'io1'

HIPCHAT_URL
The url for the HipChat API
HIPCHAT_ROOM
The HipChat room to which notifications should be delivered
HIPCHAT_TOKEN
The authentication token for the HipChat API
CURL_OPTIONS
A set of options to pass to the curl commands executed by the scripts
BITBUCKET_VERBOSE_BACKUP
If FALSE info and print level logging will be skipped

 

Setting up the instance role

The DIY backup and restore scripts use the AWS CLI toolset to do their job. These tools need to authenticate with AWS in order to gain access to your resources (EBS volumes, snapshots, etc). The recommended way of providing credentials to the instance is by launching it with an instance role that has a suitable policy attached. If you are using the Bitbucket Server CloudFormation template, it will take care of creating a policy for you and attach it to the instance at launch time.

If you need to create your own policy, you can use this JSON object as an example of the minimum permissions required for an instance:

{
    "Statement": [
        {
            "Resource": [
                "*"
            ],
            "Action": [
                "ec2:AttachVolume",
                "ec2:CreateSnapshot",
                "ec2:CreateTags",
                "ec2:CreateVolume",
                "ec2:DescribeSnapshots",
                "ec2:DescribeVolumes",
				"ec2:DetachVolume"
            ],
            "Effect": "Allow"
        }
    ],
    "Version": "2012-10-17"
}

For other ways of configuring the AWS CLI toolset, please refer to the documentation.

Last modified on Sep 21, 2015

Was this helpful?

Yes
No
Provide feedback about this article

Related content

Powered by Confluence and Scroll Viewport.