Set up a Bamboo Data Center cold standby
Bamboo Data Center allows you to run a cluster of Bamboo nodes in a cold standby configuration, providing higher availability. This guides walks you through the process of configuring a Data Center cluster on your infrastructure.
Not sure if a cold standby is right for you? Check out Running Bamboo Data Center in a cluster for a detailed overview.
Before you begin
Things you should know about when setting up your Data Center:
To set up and configure your cluster
We recommend completing this process in a staging environment, and testing your cold standby installation, before moving to production.
1. Install Bamboo Data Center on the first application node
First, you'll need make a fresh installation of Bamboo Data Center in one node following the Bamboo installation instructions.
2. Provision the shared database and filesystem
Once you've installed the first Bamboo application node, you now need to provision the share database and shared filesystem to use with Bamboo Data Center.
Step 1. Provision your shared database
Set up your shared database server.
Ensure your database is configured to allow enough concurrent connections. For example, in PostgreSQL the default limit is usually 100 connections. If you use PostgreSQL, you may need to edit your postgresql.conf
file, to increase the value of max_connections
, and restart Postgres.
Note that, while cold standby nodes are mostly idle, they periodically connect to the database in order to update their status and, possibly, decide to take over the role of the primary node.
See Connecting Bamboo Server to an external database for more information, and note that clustered databases are not supported.
Step 2. Provision your shared file system
A properly resourced and configured NFS server can perform well even under very heavy load. We’ve created some recommendations for setting up and configuring your file server for optimal performance.
If your Bamboo server is running a version older than 8.0, you’ll need to upgrade to 8.0 first. This will rearrange Bamboo server home, creating the shared subdirectory.
Once on 8.0, the shared subdirectory of the Bamboo Server home directory will contain all the configuration data, build and deployment result files, among other important files. The migration will consist of moving this folder to the Bamboo Data Center’s NFS file system and alter the path of the shared home folder inside the ${BAMBOO_INSTALLATION_FOLDER}/atlassian-bamboo/WEB-INF/classes/bamboo-init.properties
#bamboo-init.properties
bamboo.home=/your/local/home/path
bamboo.shared.home=/the/mounted/shared/filesystem/path+shared
The remaining of the subdirectories (analytics-logs, caches, export, local-working-dir,lib, logs, plugins, and temp), inside the now local home path, contain only caches, workspaces of the local agents and temporary files. You don’t need to restore them.
Provision application cluster nodes
Provision cluster node infrastructure. You can automate this using a configuration management tool such as Chef, Puppet, or Vagrant, and/or by spinning up identical virtual machine snapshots.
Step 1. Configure file share mounts
On each cluster node, mount the shared home directory as any path. We recommend using the same filesystem path on all the nodes for the sake of simplicity. Configure that path as bamboo shared home, by exporting ${BAMBOO_SHARED_HOME}
or setting bamboo.shared.home
in the bamboo-init.properties file. Alternatively, if you can skip the last step by mounting the shared directory as ${BAMBOO_HOME}/shared
, which is the default location of shared bamboo home in case it is not defined explicitly. Note that only the shared directory should be shared between cluster nodes. All other directories, including ${BAMBOO_HOME}
, should be node-local (that is, private to each node).
Example
For example, suppose your Bamboo home directory is /var/atlassian/application-data/bamboo
, and your shared home directory is available as an NFS export called bamboo-san:/bamboo-shared. To configure the mount on each cluster node:
Add the following line to
/etc/fstab
on each cluster node.
/etc/fstabbamboo-san:/bamboo-shared /var/atlassian/application-data/bamboo/shared nfs rw,nfsvers=3,lookupcache=pos,noatime,intr,rsize=32768,wsize=32768,_netdev 0 0
Mount the share on each node. Issue:
mkdir -p /var/atlassian/application-data/bamboo/shared sudo mount -a
Step 2. Synchronize system clocks
Ensure all your cluster nodes have synchronized clocks and identical timezone configuration. Here are some examples for how to do this:
Step 3. Install Bamboo Data Center on each node
On each cluster node, perform the same steps from Install Bamboo Data Center on the first application node but without making migrations or running a fresh installation. Before starting the new nodes in the latter steps, you need to make sure that the installation has the correct paths, and that the bamboo.cfg.xml from the initial node is copied inside the local home folder of this node.
If you automate the installation or clone the new cluster node, make sure that the cluster-node.properties
file doesn't exist on the new cluster node. Bamboo uses the cluster-node.properties
file to identify the nodes. Any duplicates of that file will cause multiple active nodes to start in parallel, leading to data corruption.
Step 4. Start the first cluster node
If you haven’t configured the shared home folder path yet, edit ${BAMBOO_INSTALLATION_FOLDER}/atlassian-bamboo/WEB-INF/classes/bamboo-init.properties by altering the bamboo.shared.home
property. Remember that every node will need access to this folder over the NFS.
#bamboo-init.properties
bamboo.home=/your/local/home/path
bamboo.shared.home=/the/mounted/shared/filesystem/path+shared
Install and configure your load balancer
Step 1. Configure protocols and health checks on your load balancer
Your load balancer must proxy three protocols:
Protocol | Typical port on the load balancer | Default port on the Bamboo cluster nodes | Notes |
---|---|---|---|
HTTP | 80 | 8085 | HTTP mode. Session affinity ("sticky sessions") should be enabled using the 52-character BAMBOOSESSIONID cookies. |
HTTPS | 443 | 8085 | HTTP mode. Terminating SSL at the load balancer and running plain HTTP to the Bamboo cluster nodes is highly recommended. |
TCP | 54663 | 54663 | TCP mode. For remote agents JMS connection. To prevent communication failure, the Load Balancer should be configured to SSL Passthrough the request from the remote agents to Bamboo server. |
Your load balancer must support session affinity ("sticky sessions"). Bamboo Data Center assumes that your load balancer always directs each user's requests to the same cluster node.
When choosing a load balancer, it must support the HTTP, HTTPS, and TCP protocols. Note that:
Apache doesn't support TCP mode load balancing.
HAProxy versions older than 1.5.0 do not support HTTPS.
If your load balancer supports health checks of the cluster nodes, configure it to perform a periodic HTTP GET to http://<bamboo>:8085/rest/api/latest/status
, where <bamboo>
is the cluster node's name or IP address.
If a cluster node does not return 200 OK within a reasonable amount of time, the load balancer should not direct any traffic to it.
You should then be able to navigate to http://<load-balancer>/,
where <load-balancer>
is your load balancer's name or IP address. This should take you to your Bamboo front page.