Install Bitbucket Data Center

In this guide we'll run you through installing Bitbucket Data Center, which is a clustered solution, in a Linux environment. 

This guide covers installing for the first time, with no existing data, or that you already have a production instance of Bitbucket Server, and that you are aiming to migrate that to a Bitbucket Data Center instance. 


Other ways to install Bitbucket Data Center:

Clustering requirements and terminology

To run Bitbucket in a cluster you must have a Bitbucket Data Center license, or, obtain an evaluation license.

On this page:




In this guide we'll use the following terminology:

  • Installation directory - <Bitbucket install directory> is the directory where you installed Bitbucket on a node.
  • Shared home directory – <Bitbucket shared home> The directory you created that is accessible to all nodes in the cluster via the same path.


Bitbucket Data Center component requirements

You can see a component diagram of a typical Bitbucket Data Center instance, and read about detailed requirements of each component on the page Bitbucket Data Center requirements and on the Supported platforms page.

A Bitbucket Data Center instance consists of a cluster of components, each running on a dedicated machine:

  • A cluster of Bitbucket application nodes all running the same version of Bitbucket Data Center web application. These can be virtual or physical machines, have synchronized clocks (for example, using NTP) and be configured with the identical timezone, are allowed to connect to a Bitbucket cluster node's Hazelcast port, which by default is port 5701.
  • A load balancer that supports both HTTP mode (for web traffic) and TCP mode (for SSH traffic), and support session affinity ("sticky sessions").
  • A supported external database, shared and available to all all cluster nodes.
  • A shared file system that is physically located in the same data center, available to all clusters nodes, and accessible by NFS as a single mount point.
  • A remote Elasticsearch instance with only one remote connection to Bitbucket. The instance may be a standalone Elasticsearch installation or a clustered installation behind a load balancer. For help installing and configuring a remote Elasticsearch instance see our how to guide


Be sure to test your Bitbucket Data Center installation thoroughly before deploying to production

Regardless of the installation process you use, be sure to smoke test your Bitbucket Data Center instance every step of the way. Be sure to:

  • Set up and test Bitbucket Data Center in your staging environment before deploying to a production environment.
  • Test Bitbucket Data Center with identical data (repositories, users, apps) to your production instance. 


Accessing a Bitbucket Data Center installation

At the end of the installation process, the URL of the Bitbucket Data Center instance will be the URL of the load balancer, so this is the machine that you will need to assign the name of your Bitbucket Data Center instance in the DNS. The remaining machines (Bitbucket cluster nodes, shared database, and shared file system) do not need to be publicly accessible to your users. 

Before you begin

Before installing Bitbucket Data Center, answer the following question:

Have you familiarized yourself with the requirements of Bitbucket Data Center?

You can read all about it on Bitbucket Data Center requirements.

Do you already have a production instance of Bitbucket Server running?

Yes, I have an existing production instance of Bitbucket Server running...

You need to upgrade your Bitbucket Server to the latest version. This is necessary for several reasons:

  • The Bitbucket Server database and home directory layout often change in each release of Bitbucket Server. Upgrading first will ensure that your production Bitbucket Server instance and your Bitbucket Data Center instance share identical data format, and you can switch between them at will. 
  • Any apps in your production instance can be verified as compatible with the latest release of Bitbucket Server (or updated if not). 
  • Any performance or other comparisons between single-node Bitbucket Server and multi-node Bitbucket Data Center will be more meaningful. 


To begin installing Bitbucket Data Center:

  1. Upgrade your Bitbucket Server instance to the latest release by following the instructions on the Bitbucket Server upgrade guide.
  2. Apply your Bitbucket Data Center license.
    1. Go to > Licensing > Edit license, then paste your Bitbucket Data Center license.
  3. Once you've upgraded to the latest version of Bitbucket Server, you need to stop Bitbucket and backup the database and home directory of the upgraded instance.


Take a backup of your production Bitbucket Server instance's database and home directory. For this you can:

  • use your own DIY backup solution, or
  • stop Bitbucket Server and manually dump your database, and zip up the home directory. 

You will use these backups to provision your shared file system and shared database for Bitbucket Data Center. 

Install Bitbucket Data Center on the first application node

First, you'll need to install Bitbucket Data Center one node:

  1. Download the latest installer - www.atlassian.com/software/bitbucket/download.

  2. Make the installer executable.

    Show me how to do this...

    Change to the directory where you downloaded the installer, then execute this command:

    chmod +x atlassian-bitbucket-x.x.x-x64.bin

    Where x.x.x is the version you downloaded.

  3. Run the installer – we recommend using sudo to run the installer as this will create a dedicated account to run Bitbucket and allow you to run Bitbucket as a service.

    Show me how to do this...

    To use sudo to run the installer execute this command: 

    $ sudo ./atlassian-bitbucket-x.x.x-x64.bin

    Where x.x.x is the version you downloaded.

    You can also run the installer with root user privileges.

  4. Follow the prompts to install Bitbucket. You'll be asked for the following info:
    1. Type of Bitbucket instance - the type of installation, for these instructions select Data Center.
    2. Installation directory - where Bitbucket will be installed.
    3. Home directory - where Bitbucket application data will be stored.
    4. TCP ports - the HTTP connector port and control port Bitbucket will run on.

Provision the shared database, filesystem, and Elasticsearch nodes

Once you've installed the first Bitbucket application node, you now need to provision the share database and shared filesystem to use with Bitbucket Data Center. 

Step 1. Provision your shared database

Set up your shared database server. 

Ensure your database is configured to allow enough concurrent connections. Bitbucket Server by default uses up to 80 connections per cluster node, which can exceed the default connection limit of some databases. For example, in PostgreSQL the default limit is usually 100 connections. If you use PostgreSQL, you may need to edit your postgresql.conf file, to increase the value of max_connections, and restart Postgres.

See Connecting Bitbucket Server to an external database for more information, and note that clustered databases are not supported.

You cannot use MySQL for Bitbucket Data Center...

We do not support MySQL for Bitbucket Data Center at this time due to inherent deadlocks that can occur in this database engine at high load.  If you currently use MySQL, you should migrate your data to another supported database (such as PostgreSQL) before upgrading your Bitbucket Server instance to Bitbucket Data Center. You can migrate databases (on a standalone Bitbucket Server instance) using the Migrate database feature in Bitbucket Server's Administration pages.

Step 2. Provision your shared file system

A properly resourced and configured NFS server can perform well even under very heavy load. We’ve created some recommendations for setting up and configuring your file server for optimal performance.

File server requirements

Bitbucket Data Center requires a high performance shared file system, such as a storage area network (SAN), network-attached storage (NAS), RAID server, or high-performance file server optimized for input/output.

The file system must:

  • run on a dedicated machine; avoid hosting other services on your NFS server

  • be in the same physical data center

  • be available to all Bitbucket nodes via a high-speed LAN (such as 10GB ethernet or Fibre Channel)

  • be accessible via NFS as a single mount point

  • have the NFS lock service enabled

tip/resting Created with Sketch.

For reference, our AWS Quick Start for Bitbucket Data Center deploys NFS and the Bitbucket cluster nodes on the same VPC, availability zone, and subnet.

File server user account requirements

You need to create a user account named bitbucket on the shared file system server. This user account should have read/write permissions to the shared subdirectory of the Bitbucket Server home directory.

To ensure this:

  • set bitbucket to own all files and folders in the shared subdirectory

  • create bitbucket with the user umask 0027

  • assign the same UID to bitbucket on all NFS Server and Bitbucket cluster nodes

SCM Cache plugin recommendations

Where possible, enable the SCM Cache plugin (already bundled in Bitbucket Data Center) with as much disk or SSD space on your cluster nodes as you can.

An effective SCM Cache can greatly reduce load on your shared file server. Note that the drive or partition used by the SCM Cache is local to each cluster node, not NFS mounted.

See Scaling Bitbucket Server for Continuous Integration performance for more information.

File server node recommendations

On your file server, ensure that NFS is configured with enough server processes. For example, some versions of Red Hat Enterprise Linux and CentOS have a default of 8 server processes. If you use either distribution, you may need to edit your /etc/sysconfig/nfs file, increase the value of RPCNFSDCOUNT, and restart the nfs service.

For the file server and cluster nodes, avoid kernel and NFS version combinations that are unstable or have known NFS bugs. We recommend avoiding Linux kernel versions 3.2 to 3.8.

Recommended mount options

When you provision your application cluster nodes later, we recommend using the following NFS mount options:

rw,nfsvers=3,lookupcache=pos,noatime,intr,rsize=32768,wsize=32768,_netdev

Step 4. Migrating from an existing Bitbucket Server instance (optional)

The shared subdirectory of the Bitbucket Server home directory contains all the GIT repositories, configuration data, and other important files. When migrating from an existing Bitbucket Server instance, back up shared and restore it on the new Bitbucket Data Center’s NFS file system.

The remaining subdirectories (bin, caches, export, lib, log, plugins, and tmp) contain only caches and temporary files. You don’t need to restore them.


Step 5. Provision your Elasticsearch node

To set up your Elasticsearch server, you will

  1. Install Elasticsearch on a remote machine.
  2. Configure Elasticsearch's elasticsearch.yml file to work with Bitbucket Data Center.
  3. Secure Elasticsearch with a username and password that Bitbucket will use to access Elasticsearch, with a minimum of HTTP restricted access.
  4. Connect Elasticsearch to Bitbucket.

There are detailed instructions on the page How to Install and configure a remote Elasticsearch instance to help you provision your remote Elasticsearch installation.


Provision application cluster nodes

Provision cluster node infrastructure. You can automate this using a configuration management tool such as Chef, Puppet, or Vagrant, and/or by spinning up identical virtual machine snapshots. 

Step 1. Configure file share mounts

On each cluster node, mount the shared home directory as ${BITBUCKET_HOME}/shared. Note that only the ${BITBUCKET_HOME}/shared directory should be shared between cluster nodes.  All other directories, including ${BITBUCKET_HOME}, should be node-local (that is, private to each node).  

Example

For example, suppose your Bitbucket home directory is /var/atlassian/application-data/bitbucket , and your shared home directory is available as an NFS export called bitbucket-san:/bitbucket-shared. To configure the mount on each cluster node:

  1. Add the following line to /etc/fstab on each cluster node.

    /etc/fstab
    bitbucket-san:/bitbucket-shared /var/atlassian/application-data/bitbucket/shared nfs rw,nfsvers=3,lookupcache=pos,noatime,intr,rsize=32768,wsize=32768,_netdev 0 0
  2. Mount the share on each node:

    mkdir -p /var/atlassian/application-data/bitbucket/shared
    sudo mount -a


Step 2. Synchronize system clocks

Ensure all your cluster nodes have synchronized clocks and identical timezone configuration. Here are some examples for how to do this:


For RedHat Enterprise Linux or CentOS:
sudo yum install ntp
sudo service ntpd start
sudo tzselect


For Ubuntu Linux:
sudo apt-get install ntp
sudo service ntp start
sudo dpkg-reconfigure tzdata

Step 3. Install Bitbucket Data Center on each node

On each cluster node, perform the same steps from Install Bitbucket Data Center on the first application node section.


Step 4. Start the first cluster node

Edit the file ${BITBUCKET_HOME}/shared/bitbucket.properties and add the following lines:

# Use multicast to discover cluster nodes (recommended).
hazelcast.network.multicast=true

# If your network does not support multicast, you may uncomment the following lines and substitute
# the IP addresses of some or all of your cluster nodes. (Not all of the cluster nodes have to be
# listed here but at least one of them has to be active when a new node joins.) 
#hazelcast.network.tcpip=true
#hazelcast.network.tcpip.members=192.168.0.1:5701,192.168.0.2:5701,192.168.0.3:5701

# The following should uniquely identify your cluster on the LAN. 
hazelcast.group.name=your-bitbucket-cluster
hazelcast.group.password=your-bitbucket-cluster
I'm installing in an IPv6 environment

If you are installing Bitbucket Server in an IPv6 environment, Hazelcast needs an additional system property to work.

This property is only needed if the nodes are configured to use IPv6 to talk to each other.

Edit the file ${BITBUCKET_INSTALL}/bin/_start-webapp.sh and edit the JVM_SUPPORT_RECOMMENDED_ARG line to look like this:

JVM_SUPPORT_RECOMMENDED_ARGS="-Dhazelcast.prefer.ipv4.stack=false"

The line is commented out, so be sure to remove the leading # to make the line take effect.


Using multicast to discover cluster nodes (hazelcast.network.multicast=true) is recommended, but requires all your cluster nodes to be accessible to each other via a multicast-enabled network. If your network does not support multicast then you can set  hazelcast.network.multicast=false hazelcast.network.tcpip=true, and hazelcast.network.tcpip.members to a comma-separated list of cluster nodes instead. Only enable one of hazelcast.network.tcpip or hazelcast.network.multicast, not both.

Choose a name for hazelcast.group.name and hazelcast.group.password that uniquely identifies the cluster on your LAN. If you have more than one cluster on the same LAN (for example, other Bitbucket Data Center instances or other products based on similar technology such as Confluence Data Center) then you must assign each cluster a distinct name, to prevent them from attempting to join together into a "super cluster". 

Then start Bitbucket Server. See Starting and stopping Bitbucket Server.

Then go to http://<bitbucket>:7990/admin/license, and install your Bitbucket Data Center license. Restart Bitbucket Server for the change to take effect. If you need a Bitbucket Data Center license, you can purchase one that fits your needs, or, get an evaluation license.


Install and configure your load balancer

You can use the load balancer of your choice, either hardware or software. Bitbucket Data Center does not bundle a load balancer. 

Step 1. Configure protocols and health checks on your load balancer

Your load balancer must proxy three protocols:

Protocol Typical port on the load balancer Typical port on the Bitbucket cluster nodes Notes
HTTP 80 7990 HTTP mode. Session affinity ("sticky sessions") should be enabled using the 52-character BITBUCKETSESSIONID cookie.
HTTPS 443 7990 HTTP mode. Terminating SSL at the load balancer and running plain HTTP to the Bitbucket cluster nodes is highly recommended.
SSH 7999 7999 TCP mode.

Your load balancer must support session affinity ("sticky sessions") using the BITBUCKETSESSIONID cookie. Bitbucket Data Center assumes that your load balancer always directs each user's requests to the same cluster node. If it does not, users may be unexpectedly logged out or lose other information that may be stored in their HTTP session.

When choosing a load balancer, it must support the HTTP, HTTPS, and TCP protocols. Note that:

  • Apache does not support TCP mode load balancing.
  • HAProxy versions older than 1.5.0 do not support HTTPS.

If your load balancer supports health checks of the cluster nodes, configure it to perform a periodic HTTP GET of http:// <bitbucket>:7990/status, where <bitbucket> is the cluster node's name or IP address. This returns one of two HTTP status codes:

  • 200 OK
  • 500 Internal Server Error

If a cluster node does not return 200 OK within a reasonable amount of time, the load balancer should not direct any traffic to it. 

You should then be able to navigate to http://<load-balancer>/, where <load-balancer> is your load balancer's name or IP address. This should take you to your Bitbucket Server front page. 

Example: HAProxy load balancer

If you don't have a particular preference or policy for load balancers, you can use HAProxy which is a popular Open Source software load balancer.

If you choose HAProxy, you must use a minimum version of 1.5.0. Earlier versions of HAProxy do not support HTTPS.

To check which version of HAProxy you use, run the following command:

haproxy --version

Here is an example haproxy.cfg configuration file (typically found in the location /etc/haproxy/haproxy.cfg).  This assumes:

  • Your Bitbucket cluster node is at address 192.168.0.1, listening on the default ports 7990 (HTTP) and 7999 (SSH). 
  • You have a valid SSL certificate at /etc/cert.pem.
haproxy.cfg
global
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    tune.ssl.default-dh-param 1024
defaults
    log                     global
    option                  dontlognull
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000
    errorfile               408 /dev/null	# Workaround for Chrome 35-36 bug.  See http://blog.haproxy.com/2014/05/26/haproxy-and-http-errors-408-in-chrome/

frontend bitbucket_http_frontend
    bind *:80
    bind *:443 ssl crt /etc/cert.pem ciphers RC4-SHA:AES128-SHA:AES256-SHA
    default_backend bitbucket_http_backend

backend bitbucket_http_backend
    mode http
    option httplog
    option httpchk GET /status
    option forwardfor
    option http-server-close
    appsession BITBUCKETSESSIONID len 52 timeout 1h
    balance roundrobin
    cookie BITBUCKETSESSIONID prefix
    stick-table type string len 52 size 5M expire 30m
    stick store-response set-cookie(BITBUCKETSESSIONID)
    stick on cookie(BITBUCKETSESSIONID)
    server bitbucket01 192.168.0.1:7990 check inter 10000 rise 2 fall 5
    #server bitbucket02 192.168.0.2:7990 check inter 10000 rise 2 fall 5
    # The following "backup" servers are just here to show the startup page when all nodes are starting up
    server backup01 192.168.0.1:7990 backup
    #server backup02 192.168.0.2:7990 backup

frontend bitbucket_ssh_frontend
    bind *:7999
    default_backend bitbucket_ssh_backend
    timeout client 15m
    maxconn 50

backend bitbucket_ssh_backend
    mode tcp
    balance roundrobin
    server bitbucket01 192.168.0.1:7999 check port 7999
    #server bitbucket02 192.168.0.2:7999 check port 7999
    timeout server 15m

listen admin
    mode http
    bind *:8090
    stats enable
    stats uri /

Review the contents of the haproxy.cfg file carefully, and customize it for your environment. See http://www.haproxy.org/ for more information about installing and configuring haproxy.

Once you have configured the haproxy.cfg file, start the haproxy service.

sudo service haproxy start

You can also monitor the health of your cluster by navigating to HAProxy's statistics page at http://<load-balancer>:8090/. You should see a page similar to this:

Step 2. Configure Bitbucket for HAProxy

Bitbucket needs to be configured to work with HAProxy. For example:

<Bitbucket home directory>/shared/bitbucket.properties
server.proxy-name=bitbucket.company.com
server.proxy-port=443
server.secure=true
server.require-ssl=true


Read Securing Bitbucket behind HAProxy using SSL for more details.

Step 3. Add a new Bitbucket application node to the cluster

Go to a new cluster node, and start Bitbucket Server. See Starting and stopping Bitbucket Server.

Once Bitbucket Server has started, go to https://<load-balancer>/admin/clustering. You should see a page similar to this:


Verify that the new node you have started up has successfully joined the cluster. If it does not, please check your network configuration and the ${BITBUCKET_HOME}/log/atlassian-bitbucket.log files on all nodes. If you are unable to find a reason for the node failing to join successfully, please contact Atlassian Support .


Step 4. Connect the new Bitbucket cluster node to the load balancer

If you are using your own hardware or software load balancer, consult your vendor's documentation on how to add the new Bitbucket cluster node to the load balancer.

If you are using HAProxy, uncomment these lines

server bitbucket02 192.168.0.2:7990 check inter 10000 rise 2 fall 5
server bitbucket02 192.168.0.2:7999 check port 7999

in your haproxy.cfg file and restart haproxy:

sudo service haproxy restart

Verify that the new node is in the cluster and receiving requests by checking the logs on each node to ensure both are receiving traffic and also check that updates done on one node are visible on the other. 


Congratulations!

That's it! Bitbucket Data Center is accessible from a URL like this: http://<load_balancer_IP_address>:<port>

What's next?

When setting up Bitbucket in a production environment, we recommend that you configure these aspects next:


Read more about setting up Bitbucket Server for an enterprise here: Using Bitbucket Server in the enterprise.

Last modified on Dec 10, 2019

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.