Creating a custom elastic image
Atlassian doesn't provide support for customized images. Bamboo provides flexibility to use customized machine images, but it's impossible for us to support all individual configurations.
Use Bamboo stock images as the base for all image customizations to ensure a minimal level of consistency of your Elastic Bamboo setup.
An elastic image is an Amazon Machine Image (AMI) that is stored in one of Amazon data centers for use with the Elastic Bamboo feature. An elastic image is used to create elastic instances, which in turn create elastic agents. Conceptually, an elastic image is equivalent to an operating system running on a computer's boot hard drive and elastic instances would be the software that runs on this operation system.
Each elastic image registered with the Amazon Web Services (AWS) has its own unique identifier, known as an AMI ID.
You can associate multiple elastic images with a Bamboo server. One default shared image is maintained by Atlassian in AWS, and is available to all Elastic Bamboo users.
At a high level, the process for creating a custom elastic image consists of taking one of the existing Amazon Machine Images (AMIs) available on Amazon EC2, starting an instance of the AMI, customizing the instance and then creating an image from the customized instance. This image can then be used as an elastic image in your Bamboo installation.
Instead of creating a custom image (Linux/UNIX only) consider:
- Using the "Instance setup script" feature to run commands (executed as the root user) before the agent is started. This field is available for every image from the Administration > Image Configurations page. Select the Edit link under Operations for the image you want to use.
- Customizing an existing Bamboo image by using Amazon's Elastic Block Store (EBS), as described in Configuring elastic instances to use the EBS.
The options above are much simpler than creating a new custom image. If you are having problems, please don't hesitate to contact us for further help.
Before you begin:
- This is not a trivial procedure and chances are you don't need it.
- Note that Atlassian does not support custom elastic images. Consider customizing the elastic agents started from your stock images instead.
- A number of the EC2 commands in the steps below can be completed using the AWS console rather than command line tools (e.g. registering an image). You should use the method you feel most comfortable with.
On this page:
1. Requirements
First ensure that you have set up the following:
- Amazon Web Services (AWS) account with EC2 — if you are already using Elastic Bamboo, you should already have an AWS account with EC2 set up. If not, please read Getting started with Elastic Bamboo.
Amazon EC2 API Tools — you must install the EC2 API tools on your local machine, otherwise you will not be able to start and access your AMI instance. Note: you need Java Runtime Environment to run these tools. You can install the EC2 API tools by executing the following commands:
wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip unzip ec2-api-tools.zip
- Environment Variables— you must set up the following environment variables on your local machine before creating a custom elastic image:
EC2_HOME
— set this to the path to the installed EC2 API ToolsEC2_CERT
— set this to the path to the certificate assigned to EC2 accountEC2_PRIVATE_KEY
— set this to the path to the private key assigned to your AWS account
Registered Key Pair— you need a registered EC2 key pair, which consists of a private key file and certificate file, to use the EC2 API tools with your AMI instance. If you have previously generated and registered an EC2 key pair (e.g. to use the EC2 API tools), you can re-use it. If you need to generate a new key pair, you can use the following command to do so:
ec2-add-keypair <key_pair_name>
The content of the private key will be displayed in the command-line output on your console. Save this content in a file , starting with the line:
"--BEGIN RSA PRIVATE KEY--"and ending with the line:
"--END RSA PRIVATE KEY--"This private key file will be used to access your AMI instance. Set up the appropriate permissions on the private key file by executing the following command:
chmod 600 <private_key_file>
2. Selecting an existing AMI
We strongly recommend that you select an existing Linux/UNIX AMI to customize, rather than starting with a blank AMI. When choosing an AMI, decide whether you want to launch 32-bit or 64-bit instances from your custom elastic image and select an existing AMI matching your choice.
We recommend the following existing Linux/UNIX AMIs for customization (in order of preference):
Source | Description | AMI list |
---|---|---|
Atlassian | One of the Stock images provided by Atlassian. It is an Amazon image, for either Linux or Windows, updated and prepared for Bamboo, i.e. you will not have to install any Bamboo prerequisites. | Available on your Bamboo instance under Administration/Image Configurations |
Amazon | CentOS-based image provided by Amazon. It does not have any Bamboo prerequisites installed. Typically, you will be better off using the Atlassian AMI. | Amazon's site |
Canonical(Ubuntu) | An official Ubuntu image provided by Canonical (the company behind the Ubuntu Linux project). It does not have any Bamboo prerequisites installed. | Canonical's site |
Atlassian's AMIs (and hence, their IDs) may change with each release of Bamboo, including both major and minor releases. To quickly access Atlassian's AMI IDs for your currently-running version of Bamboo, open that Bamboo site in a web browser and access its 'Image Configurations' page (see Managing your Elastic Image Configurations for more information). The AMI IDs of Atlassian's AMIs are labeled with "(stock image)".
If you want to find out the AMI IDs for a version of Bamboo you don't have running or you're starting an image from scratch and you need the image baseline:
- Open https://packages.atlassian.com/maven/repository/public/com/atlassian/bamboo/atlassian-bamboo/
- On the resulting directory page, select the link that represents the version of Bamboo you are currently running. For example, if you are running Bamboo 5.9.7, select on the 5.9.7 link. Another directory page opens, listing a .pom and some additional checksum files.
Do not select a version number link that contains 'mX', 'rcX' or 'betaX' (where 'X' is a number), since these relate to publicly available developmental releases of Bamboo. - Open the atlassian-bamboo-x.x.x.pom file (where
x.x.x
) is your version of Bamboo). The image version/baseline is stored in theelastic-image.version
tag. For example, this value is "4.2" for Bamboo version 5.9.4. - Open https://packages.atlassian.com/maven/repository/public/com/atlassian/bamboo/atlassian-bamboo-elastic-image/
- Select the image version/baseline you found in the
elastic-image.version
tag.
On the resulting directory page, the file withami
extension contains all stock image AMI IDs.
3. Starting an instance
After you have selected an existing AMI to customize, the next step is to start an instance of the AMI.
3.1 Starting an instance of Atlassian's default AMI
If you chose to customize Atlassian's default AMI, you will have to start the instance from the admin section of Bamboo. See Starting an elastic instance.
Note that Atlassian's default AMI cannot be started using the command line ec2 tools. This is because, on start up, the Bamboo agent on Atlassian's AMI checks to see if it was started from a Bamboo server, and immediately shuts itself down if it was not.
Once started, see Accessing an elastic instance for details on how to access the running instance.
3.2 Starting an instance from the command line
Use the ec2-run-instances
command to start your instance, as follows:
ec2-run-instances <image_name> -k <key_pair_name>
where <image_name>
is the name of the AMI selected in the previous step and <key_pair_name>
is the name of the registered key pair generated in '1. Requirements'. The public certificate of this key will be injected into your instance.
For example, if you wanted to start an instance of image ami-e55bbd8c
using key pair my-keypair
, you would run the following command:
ec2-run-instances ami-e55bbd8c -k my-keypair
This command would produce the following command-line output:
INSTANCE i-25b86743 ami-e55bbd8c running my-keypair
i-25b86743
is the name of the new instance in the above example. You should note down the name of your new instance, as you will need that to access your instance in the next step.
Don't forget to shut down unused instances
Please note that once you start an instance, you will be billed by Amazon for instance uptime. If you decide to abandon the setup of a custom elastic image after this step, please ensure that you shut down your instance via the AWS console.
3.3 Starting an instance from Bamboo
You can also start a fresh, uncustomized image from Bamboo and begin customization. The drawback of this approach is that you have only 40 minutes before Bamboo shuts down your instance. The advantage is that you can customize the agent in a single step (as opposed to having to customize/create image/start from Bamboo/save image again).
4. Accessing your instance
If you started the instance from Bamboo, see Accessing an elastic instance for details on how to access the running instance.
Once your instance is running, you will need to obtain the address of the instance so you can access it. To do this, use the following command:
ec2-describe-instances <instance_name>
For example, if you wanted to find the address of instance i-25b86743
, you would enter:
ec2-describe-instances i-25b86743
This command would produce the following command-line output similar to this:
RESERVATION r-790f7210 121852097033 default
INSTANCE i-25b86743 ami-e55bbd8c ec2-174-129-94-241.compute-1.amazonaws.com
domU-12-31-39-04-38-87.compute-1.internal running elasticbamboo 0 m1.small
2009-06-24T12:36:20+0000 us-east-1c aki-a71cf9ce ari-a51cf9cc
monitoring-disabled
The address of the instance in the above example is ec2-174-129-94-241.compute-1.amazonaws.com
You can then use this address to access the instance via SSH. See Accessing an elastic instance for instructions. If you are using the example command text from that document, you will need to adjust it as follows:
- replace
/opt/bamboo/home/xml-data/configuration/elasticbamboo.pk
in the example command text with the private key file you generated in '1. Requirements'. - replace
ec2-68-111-185-197.compute-1.amazonaws.com
in the example command text with the address of your instance.
5. Customizing your instance
Now that you have a running instance, customization steps heavily depend on the operating system you're using. We've prepared separate pages with Linux-specific instructions and Windows-specific instructions.
6. Creating an image of your customized instance
The process of creating a new image varies depending whether you based your image on an instance-store or EBS-root image. You can check your image type via AWS console or using ec2-describe-images.
Creating an image from EBS-root instances
See here for instructions: Amazon Tutorial
Creating an image from instance-store (S3) instances
The final step is to create an image from your customized instance. To do this, you will require the following information:
- Amazon Account Number
- Access Key ID
- Secret Access Key
- Amazon S3 bucket name that will be used to store image (if you don't have access to Amazon S3, you can sign up on this page.)
Transfer Amazon private key file and certificate to your instance
Transfer the key files to your instance by running these commands on your local machine:scp -i <private_key_file> $EC2_PRIVATE_KEY root@<instance_address>:/mnt scp -i <private_key_file> $EC2_CERT root@<instance_address>:/mnt
where <private_key_file> is the private key file from your local machine created in step 'Registered Key Pair' of 1. Requirements and the <instance_address> is the address of your instance from '4. Accessing your Instance'.
Set up EC2_HOME and JAVA_HOME environment variables
Set up these environment variables by running the following commands on your instance:export EC2_HOME=<location of your EC2 tools installation> export EC2_PRIVATE_KEY=/mnt/<ec2_private_key_file> export EC2_CERT=/mnt/<ec2_certificate_file> export JAVA_HOME=<path to JRE used to start the agent>
You can create an image of your customized instance by using the
ec2-bundle-vol
command, as follows:ec2-bundle-vol -c $EC2_CERT -k $EC2_PRIVATE_KEY -u <amazon_account_number> -p <elastic_image_name> --batch --debug
where <elastic_image_name> is the name that you want to assign to your custom image (e.g. 'CustomImage1')
Once the image is created, you need to upload it to Amazon S3 by running the command below:
ec2-upload-bundle -b <s3_bucket_name> -m /tmp/<elastic_image_name>.manifest.xml -a <access_key_id> -s <secret_access_key> --retry --debug
where <s3_bucket_name>, <access_key_id> and <secret_access_key> are the Amazon S3 bucket name, Access Key ID and Secret Access Key described previously, and <elastic_image_name> is the name that you want to assign to your custom image (e.g. 'CustomImage1').
You will then need to register your image with Amazon EC2 by using the
ec2-register
command:ec2-register <s3_bucket_name>/<elastic_image_name>.manifest.xml
where <s3_bucket_name> is the Amazon S3 bucket name described previously and <elastic_image_name> is the name that you want to assign to your custom image (e.g. 'CustomImage1'). The output of this command will show the AMI ID of your custom image.
7. Next steps
Now that you have created a custom elastic image, there are two more steps that you will need to complete before you can use it.
First, you will need to associate your custom elastic image with your Bamboo installation by creating an Elastic Image Configuration. Please note the AMI ID of your new custom image and read Managing your Elastic Image Configurations for further instructions.
Secondly, you will need to configure the capabilities of the elastic agents that will run on instances started from your image. This is done by adding the appropriate builder, JDK, Perforce and custom capabilities to your elastic image configuration, so that it reflects what your custom elastic image actually can do. For example, if you have created a custom elastic image with JDK 1.6 and Maven 2 installed, you will need to add capabilities for JDK 1.6 and Maven 2 to the elastic image configuration. Read Configuring Elastic Agent Capabilities for further instructions.
8. Need more help?
If you need more help, there are a number of resources that you can take advantage of:
- AWS Support Center — if you are having problems with any of your Amazon services, not specifically related to Bamboo, you can obtain basic support from the AWS Support Center. Note, you will need to sign up for Premium Support to get access to web/phone support.
- AWS Resource Center — the AWS Resource Center has links to online documentation, code samples and tools for AWS services.
- Bamboo Developer Forums — please feel free to discuss any useful tips or issues regarding this process in the Bamboo Developer Forums.