Set up a Bamboo Data Center warm standby

On this page

Still need help?

The Atlassian Community is here for you.

Ask the community

Bamboo Data Center allows you to run a cluster of Bamboo nodes in a warm standby configuration, providing higher availability. This guide walks you through the process of configuring a Data Center cluster on your infrastructure. 

Not sure if a warm standby is right for you? Check out Running Bamboo Data Center in a cluster for a detailed overview.

Before you begin

Things you should know about when setting up your Data Center:

Supported platforms

See our supported platforms for information on the database, Java, and operating systems you'll be able to use. These requirements are the same for Server and Data Center deployments.

Component requirements

You can see a component diagram of a typical Bamboo Data Center instance and read about the detailed requirements of each component on the  Clustering with Bamboo Data Center page

A Bamboo Data Center instance consists of a cluster of components, each running on a dedicated machine:

  • A cluster of Bamboo application nodes all running the same version of Bamboo Data Center web application. These can be virtual or physical machines, have synchronized clocks (for example, using NTP) and be configured with the identical timezone.

  • A load balancer that supports  both  HTTP mode (for web traffic)  and  TCP mode (for ActiveMQ traffic), and supports session affinity ("sticky sessions"). If the load balancer doesn’t support both modes, then configure two separate load balancers for each traffic type, e.g. Amazon ALB and Amazon ELB.

  • supported external database shared and available to all cluster nodes.

  • shared file system that is physically located in the same data center, available to all cluster nodes, and accessible by NFS as a single mount point.

    You’ll need to create a remote directory that is readable and writable by all nodes in the cluster. There are multiple ways to do this, but the simplest is to use an NFS share, which we use as an example.

Terminology

In this guide we'll use the following terminology:

  • Installation directory: The directory where you installed Bamboo.

  • Local home directory: The home or data directory stored locally on each cluster node (if Bamboo is not running in a cluster, this is simply known as the home directory).

  • Shared home directory: The directory you created that is accessible to all nodes in the cluster, preferably via the same path. If you are not running in a cluster, this directory will be present inside the home directory.

To set up and configure your cluster

We recommend completing this process in a staging environment, and testing your warm standby installation, before moving to production. 

1. Install Bamboo Data Center on the first application node

First, you'll need to make a fresh installation of Bamboo Data Center in one node following the Bamboo installation instructions.

2. Provision the shared database and filesystem

Once you've installed the first Bamboo application node, you now need to provision the share database and shared filesystem to use with Bamboo Data Center. 

Step 1. Provision your shared database

Set up your shared database server. 

Ensure your database is configured to allow enough concurrent connections. For example, in PostgreSQL the default limit is usually 100 connections. If you use PostgreSQL, you may need to edit your postgresql.conf file, to increase the value of max_connections, and restart Postgres.

Note that, while warm standby nodes are mostly idle, they periodically connect to the database in order to update their status and, possibly, decide to take over the role of the primary node.

See Connecting Bamboo Server to an external database for more information, and note that clustered databases are not supported.

Step 2. Provision your shared file system

A properly resourced and configured NFS server can perform well even under very heavy load. We’ve created some recommendations for setting up and configuring your file server for optimal performance.

File system requirements

Bamboo Data Center requires a high-performance shared file system, such as a storage area network (SAN), network-attached storage (NAS), RAID server, or high-performance file server optimized for input/output.

The file system must:

  • run on a dedicated machine; avoid hosting other services on your NFS server

  • be in the same physical data center

  • be available to all Bamboo nodes via a high-speed LAN (such as 10GB ethernet or Fibre Channel)

File server user account requirements

You need to create a user account named bamboo on the shared file system server. This user account should have read/write permissions to the shared subdirectory of the Bamboo shared home directory.

To ensure this:

  • set bamboo to own all files and folders in the shared subdirectory

  • create bamboo with the user umask 0027

  • assign the same UID to bamboo on all NFS Server and Bamboo cluster nodes

File server node recommendations

On your file server, ensure that NFS is configured with enough server processes. For example, some versions of Red Hat Enterprise Linux and CentOS have a default of 8 server processes. If you use either distribution, you may need to edit your /etc/sysconfig/nfs file, increase the value of RPCNFSDCOUNT, and restart the NFS service.

For the file server and cluster nodes, avoid kernel and NFS version combinations that are unstable or have known NFS bugs. We recommend avoiding Linux kernel versions 3.2 to 3.8.

Recommended mount options

When you provision your application cluster nodes later, we recommend using the following NFS mount options:

rw,nfsvers=3,lookupcache=pos,noatime,intr,rsize=32768,wsize=32768,_netdev

If your Bamboo server is running a version older than 8.0, you’ll need to upgrade to 8.0 first. This will rearrange Bamboo server home, creating the shared subdirectory.

Once on 8.0, the shared subdirectory of the Bamboo Server home directory will contain all the configuration data, build and deployment result files, among other important files. The migration will consist of moving this folder to the Bamboo Data Center’s NFS file system and alter the path of the shared home folder inside the <bamboo-install-dir>/atlassian-bamboo/WEB-INF/classes/bamboo-init.properties.

#bamboo-init.properties
bamboo.home=/your/local/home/path
bamboo.shared.home=/the/mounted/shared/filesystem/path+shared
Remember to back up the home folder before moving it to the new place.

The remaining of the subdirectories (analytics-logs, caches, export, local-working-dir,lib, logs, plugins, and temp), inside the now local home path, contain only caches and temporary files. You don’t need to restore them.

Provision application cluster nodes

Provision cluster node infrastructure. You can automate this using a configuration management tool such as Chef, Puppet, or Vagrant, and/or by spinning up identical virtual machine snapshots. 

Step 1. Configure file share mounts

On each cluster node, mount the shared home directory as any path. We recommend using the same filesystem path on all the nodes for the sake of simplicity. Configure that path as bamboo shared home, by exporting ${BAMBOO_SHARED_HOME}  or setting bamboo.shared.home  in the bamboo-init.properties  file. Alternatively, if you can skip the last step by mounting the shared directory as ${BAMBOO_HOME}/shared , which is the default location of shared bamboo home in case it is not defined explicitly. Note that  only  the shared directory should be shared between cluster nodes. All other directories, including ${BAMBOO_HOME} , should be node-local (that is, private to each node). 


Example

For example, suppose your Bamboo home directory is /var/atlassian/application-data/bamboo , and your shared home directory is available as an NFS export called bamboo-san:/bamboo-shared. To configure the mount on each cluster node:

  1. Add the following line to /etc/fstab on each cluster node.
    /etc/fstab

    bamboo-san:/bamboo-shared /var/atlassian/application-data/bamboo/shared nfs rw,nfsvers=3,lookupcache=pos,noatime,intr,rsize=32768,wsize=32768,_netdev 0 0
  2. Mount the share on each node. Issue:

    mkdir -p /var/atlassian/application-data/bamboo/shared
    sudo mount -a

Step 2. Synchronize system clocks

Ensure all your cluster nodes have synchronized clocks and identical timezone configuration. Here are some examples for how to do this:

For RedHat Enterprise Linux or CentOS
sudo yum install ntp
sudo service ntpd start
sudo tzselect
For Ubuntu Linux
sudo apt-get install ntp
sudo service ntp start
sudo dpkg-reconfigure tzdata

Step 3. Install Bamboo Data Center on each node

On each cluster node, perform the same steps from Install Bamboo Data Center on the first application node but without making migrations or running a fresh installation. Before starting the new nodes in the latter steps, you need to make sure that the installation has the correct paths, and that the bamboo.cfg.xml from the initial node is copied inside the local home folder of this node.

If you automate the installation or clone the new cluster node, make sure that the cluster-node.properties file isn't a direct copy from the previous cluster node. Bamboo uses the cluster-node.properties file to identify nodes. Any duplicates of the node.id or node.name properties will cause multiple active nodes to start in parallel, leading to data corruption.

Ensure that the node.hostname is the node-specific hostname or IP address under which the node is reachable from other nodes in the cluster. You might also want to change the node.internal.communication.port properties (by default, 9090). The port must be reachable from the other nodes, but it doesn’t have to be reachable from the outside world.

Additionally, gRPC communication isn't proxied by default. If you want the underlying gRPC communication to use a proxy, set the bamboo.enable.grpc.via.proxy property to true and remove any references to your Bamboo nodes from the http.nonProxyHosts property.

Step 4. Start the first cluster node

If you haven’t configured the shared home folder path yet, edit  ${BAMBOO_INSTALLATION_FOLDER}/atlassian-bamboo/WEB-INF/classes/bamboo-init.properties by altering the bamboo.shared.home property. Remember that every node will need access to this folder over the NFS.

#bamboo-init.properties
bamboo.home=/your/local/home/path
bamboo.shared.home=/the/mounted/shared/filesystem/path+shared

Install and configure your load balancer

Step 1. Configure protocols and health checks on your load balancer

Your load balancer must proxy three protocols:

Protocol

Typical port on the load balancer

Default port on the Bamboo cluster nodes

Notes

HTTP

80

8085

HTTP mode. Make sure to enable session affinity ("sticky sessions").

HTTPS

443

8085

HTTP mode. Terminating SSL at the load balancer and running plain HTTP to the Bamboo cluster nodes is highly recommended.

TCP

54663

54663

TCP mode. For remote agents JMS connection.

To prevent communication failure, the Load Balancer should be configured to SSL Passthrough the request from the remote agents to Bamboo server.

Your load balancer must support session affinity ("sticky sessions"). Bamboo Data Center assumes that your load balancer always directs each user's requests to the same cluster node.


When choosing a load balancer, it must support the HTTP, HTTPS, and TCP protocols. Note that:

  • Apache doesn't support TCP mode load balancing.

  • HAProxy versions older than 1.5.0 do not support HTTPS.

  • If you don't have a particular preference or policy for load balancers, you can use HAProxy, which is a popular open-source load balancer. For more information, go to Configuring the HAProxy load balancer.

If your load balancer supports health checks of the cluster nodes, configure it to perform a periodic HTTP GET to http://<bamboo-url>:8085/rest/api/latest/status, where <bamboo-url> is the cluster node's name or IP address.

If a cluster node does not return 200 OK within a reasonable amount of time, the load balancer should not direct any traffic to it. 

You should then be able to navigate to http://<load-balancer>/, where <load-balancer> is your load balancer's name or IP address. This should take you to your Bamboo front page. 

Last modified on Mar 5, 2024

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.