This documentation relates to an earlier version of Bamboo.
View

Unknown macro: {spacejump}

or visit the current documentation home.

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Current »

Amazon Elastic Block Store (EBS) provides persistent storage volumes that can be attached to EC2 instances. Elastic Bamboo can use the EBS to store snapshots of relatively static build information, such as checkouts of source code and Maven repository data. You can choose a snapshot to create EBS volumes from and associate it with an elastic image configuration. These volumes can then be attached to your elastic instances when they are started from their elastic image.

Why should I use Amazon EBS with Elastic Bamboo?

Elastic agents are effectively stateless. This means that any externally-retrieved resources such as Maven artifacts, cached application servers, etc, will be downloaded again each time an elastic agent is started. The elastic agent will also perform a full checkout the first time a plan is built on the agent. If your builds have external dependencies and/or you are not performing clean builds each time, you may find that Amazon EBS will significantly improve your build times.

Additionally, EBS can be used as an easy mechanism for customising elastic agents, rather than changing the configuration of the elastic images (see Creating a Custom Elastic Image) that the agents are started from. For example, you could upload files and scripts to your EBS volume that would install PostgreSQL databases on elastic agents when are started from the related elastic instance.

On this page:

Creating your first EBS snapshot

To create your first EBS snapshot,

  1. Download Amazon Web Services (AWS) account identifiers to your Bamboo server — You will need to store the AWS private key file and certificate file on your Bamboo server to use Elastic Bamboo with EBS. If you haven't downloaded an AWS private key file or certificate file to your Bamboo server yet, please see Generating your AWS Private Key File and Certificate File for instructions.
  2. Update your Bamboo configuration settings with the location of the AWS account identifier files you have downloaded. This will ensure that these files are uploaded to any new elastic instances started. See the Elastic Instance Settings section on the Configuring Elastic Bamboo for instructions (you will need to update the 'Upload AWS account identifiers to new elastic instances (mandatory if EBS Snapshot ID specified)' checkbox and 'Account Private Key File' and 'Account Certificate File' fields described in this document).
  3. Start a single elastic instance via Bamboo. See Starting an Elastic Instance for instructions.
  4. Access your elastic instance via SSH (see Accessing an Elastic Instance for instructions).
  5. Follow the steps below to create an EBS volume and attach it to the elastic instance (step a & b), upload content to the EBS volume (step c & d), and generate the snapshot (step e & f):
    (info) All the scripts described below are bundled with Bamboo.
    1. Run createInitialVolume.sh <volume size> — This script creates a EBS volume (where <volume size> is the size of the volume), attaches the volume and mounts it on the elastic instance. For example, createInitialVolume.sh 100 will create a 100GB EBS volume and attach and mount it on the elastic instance.
    2. Run rewarmEbsSnapshot.sh — This script sets up the standard structure for Elastic Bamboo on the EBS volume. The directories and files for this standard volume structure are detailed in the Important EBS Directories and Files section below.
    3. (optional) Populate your EBS volume — Your EBS volume can now be populated with any files and scripts that you wish to make available to the elastic instances that use the EBS volume. For example, you may want to upload maven repository data, source code, scripts and files to install databases on your elastic agents, etc. You must upload your files to the /mnt/bamboo-ebs folder or its subfolders, if you want them to be included in the snapshot. We recommend that you read Populating your EBS volume for guidelines on how to populate your EBS volume effectively.
      (info) The EBS volume is attached to the elastic instance, so accessing your elastic instance via SSH will give you full access to the EBS volume (see Important EBS Directories and Files below).
    4. Ensure all uploaded content has the owner bamboo:bamboo — You can set the owner of a file by executing the following command: chown -R bamboo:bamboo <filename>
    5. Execute the killall java command — This command kills all processes on the instance, such as agent processes, so that the volume can be unmounted to be snapshotted.
    6. Run generateSnapshot.sh — This script unmounts and detaches the volume, before creating a snapshot based on the volume. The time taken to create the snapshot will vary depending on the amount of content that you have uploaded to the EBS volume. The Snapshot ID for the snapshot will be available in the logs for the elastic instance. See Accessing an Elastic Instance for instructions on how to access the logs for your elastic instance.
  6. Shut down your elastic instance. See Shutting down an Elastic Instance for instructions.

Congratulations! You have created your first EBS snapshot!

Configuring an Elastic Image to use an EBS snapshot

Once you have set up an EBS snapshot, the final step is to add the snapshot details to an elastic image configuration, so that any instances started from that image will have EBS volumes attached to them. You can associate different snapshots with different elastic image configurations.

To configure Elastic Bamboo to use an EBS snapshot,

  1. Determine the Snapshot ID of the EBS snapshot you have just created. The Snapshot ID should be recorded in the logs of the elastic instance you created it on. You can also view your EBS snapshots in the AWS Console by clicking the 'Snapshots' menu item.
    Screenshot: AWS Console (click to view full-size image)
  2. Navigate to the Bamboo administration menu, i.e. click the 'Administration' link in the top navigation bar of the Bamboo application.
  3. Click the 'Image Configurations' link in the left navigation column under the 'Elastic Bamboo' sub-header. The 'Manage Elastic Image Configurations' page will display.
  4. Click the 'Edit' link in the 'Operations' column of the elastic image configuration that you would like to add your EBS snapshot to. The 'Edit Elastic Image Configuration - <imagename>' screen will display (see screenshot below).
    Screenshot: Adding EBS Snapshot details to an Elastic Image Configuration (click to view full-size image)
  5. Check the 'Automatically attach an Amazon Elastic Block Store (EBS) volume to new elastic instances' checkbox. The 'EBS Snapshot ID' field will display.
  6. Enter the Snapshot ID of your EBS snapshot in the 'EBS Snapshot ID' field.
  7. Click the 'Save' button. A new EBS volume will be created from the specified snapshot and attached to any new elastic instances started from that image.

Updating your EBS snapshot

If you are currently using EBS with Elastic Bamboo and want to update your snapshot, follow the instructions below. These are similar to the instructions for creating a new EBS snapshot.

To update your EBS snapshot,

  1. Start a single elastic instance via Bamboo. See Starting an Elastic Instance for instructions.
  2. (optional) Run a build on the elastic agent of the instance to populate the attached EBS volume. We recommend that you read Populating your EBS volume for guidelines on how to populate your EBS volume effectively.
  3. Access your elastic instance via SSH (see Accessing an Elastic Instance for instructions) and do the following:
    (info) All the scripts described below are bundled with Bamboo.
    1. (optional) Upload any additional content to the attached EBS volume via Secure Copy (SCP). You must upload your files to the /mnt/bamboo-ebs folder or its subfolders, if you want them to be included in the snapshot.
    2. Execute killall java — This command kills all agent processes, so that nothing is using the mounted volume.
    3. Execute jps -vl — This command displays a list of all java processes running on your instance. There should be no java processes running.
    4. Run generateSnapshot.sh — This script unmounts and detaches the volume, before creating a snapshot based on the volume.
    5. Check the elastic instance logs for the Snapshot ID of the snapshot you just created. See Accessing an Elastic Instance for instructions on how to access the logs for your elastic instance.
    6. Update the new Snapshot ID in your Elastic Bamboo configuration, as described in Configuring an Elastic Image to use an EBS snapshot above.

Important EBS Directories and Files

By convention, Bamboo will attach an EBS device at /dev/sdh. This will be mounted at /mnt/bamboo-ebs. The contents of the standard structure are:

  • bin/customiseInstance.sh - This script is run on startup of an elastic instance. We recommend that you do not customise this script, as it is overwritten when rewarmEbsSnapshot.sh is run.
  • bin/customise-extras.sh - This script is run on startup of an elastic instance as the root (as opposed to being run as the Bamboo user). This script is safe to customise, as it will never be overwritten. You can customise this script to automate processes such as setting up your database, move files to custom locations on the instance, etc.
  • profile-extras.sh - This script gets appended to the profile that is run under the Bamboo user (as opposed to being run as the root). It is useful for setting up environment variables.
  • bamboo-agent/bamboo-agent.cfg.xml - This configuration file modifies the build working directory to point to build working directory on the EBS volume.
  • bamboo-agent/build-dir - This is the build working directory.
  • maven/build.properties - This properties file is copied to /home/bamboo on startup of an elastic instance. It points the Maven 1 default repository to /mnt/bamboo-ebs/maven/.maven
  • maven/settings.xml - This configuration files is copied to /home/bamboo/.m2 on startup of an elastic instance. It points the Maven 2 default repository to /mnt/bamboo-ebs/maven/.m2/repository.
  • tmp-extras - The contents of this directory is copied to /tmp on startup of an elastic instance.
  • No labels



Log a request with our support team.



Raise an issue for our developers.



See answers from the community.



Tweet, blog and update our documentation.

Except where otherwise noted, content in this space is licensed under a Creative Commons Attribution 2.5 Australia License.