Set up a Bamboo Data Center cold standby

On this page

Still need help?

The Atlassian Community is here for you.

Ask the community

Bamboo Data Center allows you to run a cluster of Bamboo nodes in a cold standby configuration, providing higher availability. This guides walks you through the process of configuring a Data Center cluster on your infrastructure. 

Not sure if a cold standby is right for you? Check out Running Bamboo Data Center in a cluster for a detailed overview.

Before you begin

Things you should know about when setting up your Data Center:

Supported platforms
See our supported platforms for information on the database, Java, and operating systems you'll be able to use. These requirements are the same for Server and Data Center deployments.
Component requirements

You can see a component diagram of a typical Bamboo Data Center instance, and read about detailed requirements of each component on the Bamboo Data Center requirements page

A Bamboo Data Center instance consists of a cluster of components, each running on a dedicated machine:

  • A cluster of Bamboo application nodes all running the same version of Bamboo Data Center web application. These can be virtual or physical machines, have synchronized clocks (for example, using NTP) and be configured with the identical timezone.

  • A load balancer that supports both HTTP mode (for web traffic) and TCP mode (for ActiveMQ traffic), and support session affinity ("sticky sessions"). If the load balancer doesn’t support both modes, then configure two separate load balancers for each traffic type, e.g. Amazon ALB and Amazon ELB.

  • supported external database, shared and available to all all cluster nodes.

  • shared file system that is physically located in the same data center, available to all clusters nodes, and accessible by NFS as a single mount point.

    You’ll need to create a remote directory that is readable and writable by all nodes in the cluster. There are multiple ways to do this, but the simplest is to use an NFS share, which we use as an example.

Terminology

In this guide we'll use the following terminology:

  • Installation directory: The directory where you installed Bamboo.

  • Local home directory: The home or data directory stored locally on each cluster node (if Bamboo is not running in a cluster, this is simply known as the home directory).

  • Shared home directory: The directory you created that is accessible to all nodes in the cluster, preferably via the same path. If you are not running in a cluster, this directory will be present inside the home directory.

To set up and configure your cluster

We recommend completing this process in a staging environment, and testing your cold standby installation, before moving to production. 

1. Install Bamboo Data Center on the first application node

First, you'll need make a fresh installation of Bamboo Data Center in one node following the Bamboo installation instructions.

2. Provision the shared database and filesystem

Once you've installed the first Bamboo application node, you now need to provision the share database and shared filesystem to use with Bamboo Data Center. 

Step 1. Provision your shared database

Set up your shared database server. 

Ensure your database is configured to allow enough concurrent connections. For example, in PostgreSQL the default limit is usually 100 connections. If you use PostgreSQL, you may need to edit your postgresql.conf file, to increase the value of max_connections, and restart Postgres.

Note that, while cold standby nodes are mostly idle, they periodically connect to the database in order to update their status and, possibly, decide to take over the role of the primary node.

See Connecting Bamboo Server to an external database for more information, and note that clustered databases are not supported.


Step 2. Provision your shared file system

A properly resourced and configured NFS server can perform well even under very heavy load. We’ve created some recommendations for setting up and configuring your file server for optimal performance.

File system requirements

Bamboo Data Center requires a high performance shared file system, such as a storage area network (SAN), network-attached storage (NAS), RAID server, or high-performance file server optimized for input/output.

The file system must:

  • run on a dedicated machine; avoid hosting other services on your NFS server

  • be in the same physical data center

  • be available to all Bamboo nodes via a high-speed LAN (such as 10GB ethernet or Fibre Channel)

File server user account requirements

You need to create a user account named bamboo on the shared file system server. This user account should have read/write permissions to the shared subdirectory of the Bamboo shared home directory.

To ensure this:

  • set bamboo to own all files and folders in the shared subdirectory

  • create bamboo with the user umask 0027

  • assign the same UID to bamboo on all NFS Server and Bamboo cluster nodes

File server node recommendations

On your file server, ensure that NFS is configured with enough server processes. For example, some versions of Red Hat Enterprise Linux and CentOS have a default of 8 server processes. If you use either distribution, you may need to edit your /etc/sysconfig/nfs file, increase the value of RPCNFSDCOUNT, and restart the NFS service.

For the file server and cluster nodes, avoid kernel and NFS version combinations that are unstable or have known NFS bugs. We recommend avoiding Linux kernel versions 3.2 to 3.8.

Recommended mount options

When you provision your application cluster nodes later, we recommend using the following NFS mount options:

rw,nfsvers=3,lookupcache=pos,noatime,intr,rsize=32768,wsize=32768,_netdev

If your Bamboo server is running a version older than 8.0, you’ll need to upgrade to 8.0 first. This will rearrange Bamboo server home, creating the shared subdirectory.

Once on 8.0, the shared subdirectory of the Bamboo Server home directory will contain all the configuration data, build and deployment result files, among other important files. The migration will consist of moving this folder to the Bamboo Data Center’s NFS file system and alter the path of the shared home folder inside the ${BAMBOO_INSTALLATION_FOLDER}/atlassian-bamboo/WEB-INF/classes/bamboo-init.properties

#bamboo-init.properties
bamboo.home=/your/local/home/path
bamboo.shared.home=/the/mounted/shared/filesystem/path+shared
Remember to back up the home folder before moving it to the new place.

The remaining of the subdirectories (analytics-logs, caches, export, local-working-dir,lib, logs, plugins, and temp), inside the now local home path, contain only caches, workspaces of the local agents and temporary files. You don’t need to restore them.

Provision application cluster nodes

Provision cluster node infrastructure. You can automate this using a configuration management tool such as Chef, Puppet, or Vagrant, and/or by spinning up identical virtual machine snapshots. 

Step 1. Configure file share mounts

On each cluster node, mount the shared home directory as any path. We recommend using the same filesystem path on all the nodes for the sake of simplicity. Configure that path as bamboo shared home, by exporting ${BAMBOO_SHARED_HOME} or setting bamboo.shared.home in the bamboo-init.properties file. Alternatively, if you can skip the last step by mounting the shared directory as ${BAMBOO_HOME}/shared, which is the default location of shared bamboo home in case it is not defined explicitly. Note that only the shared directory should be shared between cluster nodes. All other directories, including ${BAMBOO_HOME}, should be node-local (that is, private to each node). 


Example

For example, suppose your Bamboo home directory is /var/atlassian/application-data/bamboo , and your shared home directory is available as an NFS export called bamboo-san:/bamboo-shared. To configure the mount on each cluster node:

  1. Add the following line to /etc/fstab on each cluster node.
    /etc/fstab

    bamboo-san:/bamboo-shared /var/atlassian/application-data/bamboo/shared nfs rw,nfsvers=3,lookupcache=pos,noatime,intr,rsize=32768,wsize=32768,_netdev 0 0
  2. Mount the share on each node. Issue:

    mkdir -p /var/atlassian/application-data/bamboo/shared
    sudo mount -a

Step 2. Synchronize system clocks

Ensure all your cluster nodes have synchronized clocks and identical timezone configuration. Here are some examples for how to do this:

For RedHat Enterprise Linux or CentOS
sudo yum install ntp
sudo service ntpd start
sudo tzselect
For Ubuntu Linux
sudo apt-get install ntp
sudo service ntp start
sudo dpkg-reconfigure tzdata

Step 3. Install Bamboo Data Center on each node

On each cluster node, perform the same steps from Install Bamboo Data Center on the first application node but without making migrations or running a fresh installation. Before starting the new nodes in the latter steps, you need to make sure that the installation has the correct paths, and that the bamboo.cfg.xml from the initial node is copied inside the local home folder of this node.

Step 4. Start the first cluster node

If you haven’t configured the shared home folder path yet, edit  ${BAMBOO_INSTALLATION_FOLDER}/atlassian-bamboo/WEB-INF/classes/bamboo-init.properties by altering the bamboo.shared.home property. Remember that every node will need access to this folder over the NFS.

#bamboo-init.properties
bamboo.home=/your/local/home/path
bamboo.shared.home=/the/mounted/shared/filesystem/path+shared



Install and configure your load balancer

Step 1. Configure protocols and health checks on your load balancer

Your load balancer must proxy three protocols:

Protocol

Typical port on the load balancer

Default port on the Bamboo cluster nodes

Notes

HTTP

80

8085

HTTP mode. Session affinity ("sticky sessions") should be enabled.

HTTPS

443

8085

HTTP mode. Terminating SSL at the load balancer and running plain HTTP to the Bamboo cluster nodes is highly recommended.

TCP

54663

54663

TCP mode.

Agents (ActiveMQ) use TCP to connect to Bamboo nodes.

Your load balancer must support session affinity ("sticky sessions"). Bamboo Data Center assumes that your load balancer always directs each user's requests to the same cluster node.


When choosing a load balancer, it must support the HTTP, HTTPS, and TCP protocols. Note that:

  • Apache doesn't support TCP mode load balancing.

  • HAProxy versions older than 1.5.0 do not support HTTPS.

If your load balancer supports health checks of the cluster nodes, configure it to perform a periodic HTTP GET to http://<bamboo>:8085/rest/api/latest/status, where <bamboo> is the cluster node's name or IP address.


If a cluster node does not return 200 OK within a reasonable amount of time, the load balancer should not direct any traffic to it. 

You should then be able to navigate to http://<load-balancer>/, where <load-balancer> is your load balancer's name or IP address. This should take you to your Bamboo front page. 

Last modified on Dec 14, 2021

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.