Bitbucket Data Center FAQ

This page covers some of the most frequently asked questions we hear at Atlassian about Bitbucket Data Center. If you have questions that aren't answered by this page, please contact your Atlassian Enterprise Advocate, Atlassian's Enterprise team, or your Atlassian Expert.

What is Bitbucket Data Center?

How is Bitbucket Data Center different from Bitbucket Server?

Bitbucket Data Center provides all the functionality of Bitbucket Server, plus the following features:

  1. Clustering: The ability to run your Bitbucket instance on a cluster of multiple nodes (connected by a high bandwidth, low latency network), providing scalable capacity, performance, and high availability.  See Clustering with Bitbucket Data Center.
  2. Smart Mirroring: The ability to provide local Mirror nodes in geographically distributed locations, to accelerate Git clones and fetches for remote teams.  See Smart Mirroring.
  3. Disaster Recovery: The ability to maintain a cold-standby instance with a data backup or replication plan and recovery process. See Disaster recovery.

Does Bitbucket Data Center provide disaster recovery at an off-site location?

Yes. Bitbucket Data Center can be configured to implement and manage a disaster recovery strategy. Atlassian provides guidance for setting up and deploying a cold standby instance that can be configured to continuously backup your data. See the Disaster recovery guide for Bitbucket Data Center for guidance on including Bitbucket Data Center as part of your disaster recovery plan.

What kind of support is included with Bitbucket Data Center?

Support for Bitbucket Data Center is similar to existing support for Bitbucket Server.

Enterprise customers, however, may also choose to take advantage of other offerings, such as Premier Support and Technical Account Management. Contact your Atlassian Enterprise Advocate, sales@atlassian.com, or your Atlassian Expert for more information about these services.

Does Bitbucket Data Center support Windows?

Not at this time. Please contact us if you have a particular desire to do so.

Where can I download the Bitbucket Data Center distribution?

From https://www.atlassian.com/software/bitbucket/download, the same location as Bitbucket Server. Bitbucket Data Center and Bitbucket Server are in fact the same build of Bitbucket, the Clustering and Smart Mirroring functionality just needs a Data Center license key to enable.

Clustering

What are the benefits of a clustered Bitbucket Data Center instance?

Bitbucket Server is a single instance of Bitbucket running on a single machine. Its capacity to handle load is limited by the resources of a single machine, and if the machine goes down for any reason (for example, hardware failure, network fault, or planned maintenance), then Bitbucket is unavailable to users for the duration of the downtime. 

Bitbucket Data Center, on the other hand, appears to users as a single instance of Bitbucket, but "under the hood" is actually a cluster of multiple machines ("cluster nodes") each running the Bitbucket web application, behind a load balancer. This provides important benefits over a single node instance:

  • Performance at scale: A cluster of many machines each running Bitbucket can handle a greater load than a single machine.

  • High availability: If one cluster node goes down, then the remaining cluster node(s) can continue servicing requests so users see little or no loss of availability. 

  • Instant scalability: You can rapidly provision extra capacity without downtime.

For more information see Clustering with Bitbucket Data Center

There are lots of different kinds of "clustering" out there. Which one is Bitbucket Data Center, exactly?

Bitbucket Data Center is a full ("active-active") cluster configuration. All cluster nodes participate equally in servicing user requests and other processing. Load is balanced fairly between the nodes. If one node goes down, failover to the other nodes happens automatically without manual intervention, typically within seconds. When a node joins (or re-joins) the cluster, it begins servicing user requests and other processing automatically, as soon as it comes online. 

"Active-active" clustering is the key to the performance at scale and high availability advantages of Bitbucket Data Center. This is not the same as "active-passive" and other forms of clustering where data must be replicated to "cold standby" servers. 

Can a Bitbucket Data Center instance have cluster nodes in more than one geographic location, such as the Americas, Europe, and the Asia-Pacific region?

No. A Bitbucket Data Center cluster is designed to support cluster nodes in a single geographic location.

You can, though, use Smart Mirroring to provide instances in different geographic locations that accelerate Git clones and fetches (typically the longest running operations) for your remote teams.

Which add-ons or plugins are compliant with Bitbucket Data Center?

The best and most up to date way to find out whether a plugin is supported in Bitbucket Data Center is to check for the "Data Center" badge on the corresponding Atlassian Marketplace listing.

I've developed a Bitbucket add-on, how do I make it compatible with Bitbucket Data Center?

Some add-ons may not need any development work at all, and will largely "just work" in Bitbucket Data Center without modification. But any add-on that assumes (implicitly or explicitly) that it is running in a single JVM may need some development work to be made compatible. The Bitbucket API for plugin developers includes a number of new services that make it easy to write cluster-safe add-ons. These services generally provide similar API's to standard Java API's such as ConcurrentMapCache, and  Lock, so in many cases an add-on can be made cluster safe by just changing a few classes.  See Making cluster-safe plugins for more information.

Do integrations with other Atlassian products, such as JIRA and Bamboo, need special configuration in Bitbucket Data Center?

No. All integrations with other Atlassian products, including JIRA and Bamboo, work just as they do with Bitbucket Server.

Clustering performance at scale

How many cluster nodes should I deploy for my organization?

You can review our Bitbucket Data Center Performance page for some guidelines, and you can contact an Enterprise Expert Partner for a more specific recommendation for your environment. The most thorough way to determine this, however, is with your own test environment: start with two cluster nodes and upgrade to more as necessary.

Won't NFS be the bottleneck in my Bitbucket Data Center instance?

It can be, especially if your file server is under-resourced or misconfigured. But in Atlassian's own Bitbucket Data Center Performance  testing, we have found that a properly resourced and configured NFS server can perform well even under very heavy load. Here are some guidelines and recommendations when setting up your file server for Bitbucket Data Center:

  • Investing in a  high performance file server such as a SAN, NAS, or RAID server is highly recommended. 

  • Use a high-speed LAN such as 10 GB Ethernet or Fibre Channel within your cluster. 

  • The file server should run on a dedicated machine. Avoid multi-tenanting your Bitbucket Data Center file server with other services on the same physical machine. 

  • Where possible, enable the SCM Cache plugin (already bundled in Bitbucket Data Center) with as much disk or SSD space on your cluster nodes as you can. An effective SCM Cache can greatly reduce load on your shared file server. Note that the drive or partition used by SCM Cache is local to each cluster node, not NFS mounted. See Scaling Bitbucket Server for Continuous Integration performance for more information. 

  • For Linux-based file servers, ensure NFS is configured with enough server processes. For example, some versions of RedHat Enterprise Linux and CentOS have a default of 8 server processes. If you use one of these systems, you may need to edit your /etc/sysconfig/nfs file, increase the value of RPCNFSDCOUNT, and restart the nfs service.

  • For Linux-based file servers and cluster nodes, avoid kernel and NFS version combinations that are unstable or have known bugs relating to NFS. We recommend:

    • avoiding Linux kernels from about version 3.2 to 3.8 inclusive,

    • using the NFS mount options lookupcache=pos,noatime,intr,rsize=32768,wsize=32768.

See Installing Bitbucket Data Center for more information. 

Clustering high availability

What's the minimum number of cluster nodes I should use?

In order for hot failover to be possible, we recommend that you have a minimum of two cluster nodes. Within Atlassian, we've tested using up to 12 cluster nodes.

How quickly does failover work in Bitbucket Data Center?

Because Bitbucket Data Center provides full active-active clustering, failover if a node goes down can happen in seconds. You do not have to wait for a "cold standby" to start up, for all the data to replicate consistently, or for traffic redirection to propagate in the DNS. All this is handled automatically and seamlessly by Bitbucket Data Center, without intervention by the system administrator. For more detailed information, see Failover for Bitbucket Data Center.

If a cluster node goes down, what happens to users who are logged in to that node?

If a cluster node goes down, most load balancers are able to detect the failure and direct traffic to the other nodes within seconds. Users who were actively using that cluster node when it went down, however, may notice the failure in a number of ways:

  • Any open Git clone, fetch, or push connections being handled by the failing node may be broken (but will succeed if retried). 

  • Any users whose session was directed to the failing node will be logged out (by default), and will have to log in to Bitbucket again.  This extra login, though, can be avoided in a few ways:
    • Users who check "Keep me logged in" when logging in should not have to log in again. 
    • If you have enabled Single sign-on (SSO) with Crowd, then users whose Crowd directory has been configured for SSO should not have to log in again. See Connecting Bitbucket Server to Crowd for more information. 
    • If you have your own custom (in house or third party) authentication plug-in, then you can avoid the extra login provided your plug-in is written to be cluster aware. 
  • Any other data stored in the session (such as fields entered in one page of a multi-page form) on the failing node may be lost and have to be re-entered. Bitbucket and most Atlassian-supplied add-ons do not store any such state in the session, but third party add-ons that were not designed for Bitbucket Data Center may. 

  • Any background work running on the failing node, such as pull request rescopes, comment drift calculations, or user directory synchronization, may take a bit longer to complete, as this work is taken over by another cluster node. 

For more detailed information, see Failover for Bitbucket Data Center.

Can Bitbucket Data Center be upgraded without downtime (i.e., a rolling upgrade)?

No. Bitbucket Data Center does not allow cluster nodes with mixed versions in the same cluster. So even if you upgrade nodes one at a time, there must always be a period of downtime between the old and new versions. But, properly managed, upgrades of a Bitbucket Data Center instance should take no more downtime than with an equivalent Bitbucket Server instance, and can be considerably less. In Bitbucket Data Center, nodes can be taken offline one by one to perform the upgrade, leaving only the shutdown time of the last node on the old version and the startup time of the first node on the new version inside the downtime window. In Bitbucket Server instances this is only possible if you have provisioned an identical backup server for the purpose.

Clustering technical requirements

How do I move from my Bitbucket Server instance to Bitbucket Data Center?

Simply follow the steps in Installing Bitbucket Data Center. Don't forget to update your license key to a Bitbucket Data Center key.

Does Bitbucket Data Center clustering support non-multicast-enabled networks?

Yes. Multicast discovery is one common method for cluster nodes to find each other, but if your network does not support multicast, then you can set  hazelcast.network.tcpip=true  in your bitbucket.properties file, and  hazelcast.network.tcpip.members  to a comma-separated list of cluster nodes instead. See Installing Bitbucket Data Center.

Does Bitbucket Data Center clustering support CIFS/SMB, iSCSI, GlusterFS, or other file system mount types?

Not at this time. Please contact us

Does Bitbucket Data Center clustering support Amazon Web Services (AWS)?

Yes! Atlassian and Amazon even publish a standard, automated reference deployment for Bitbucket Data Center. This includes a Quick Start guide and AWS CloudFormation template that launches, configures, and runs the AWS compute, network, storage, and other services required for a complete production deployment of Bitbucket Data Center in AWS, using AWS best practices for security and availability. See Amazon Quick Starts and Bitbucket Data Center in AWS.

Does Bitbucket Data Center clustering support Amazon EFS?

Not at this time. As noted in Amazon EFS Performance:

The distributed nature of Amazon EFS […] results in a small latency overhead for each file operation. Due to this per-operation latency, overall throughput generally increases as the average I/O size increases, because the overhead is amortized over a larger amount of data.

Git typically involves many thousands of small file operations and the additional latency for each file operation means git is not suited to a distributed file system such as EFS. Since EFS's Max I/O option has an even higher latency, it is also not suitable for Bitbucket Data Center.

Which load balancers are supported by Bitbucket Data Center?

There are too many vendors providing load balancers (either hardware or software) for Atlassian to provide specific guidance on all of them. You can choose any load balancer you like, provided it supports:

  • both HTTP mode (for web traffic) and TCP mode (for ssh traffic),
  • SSL termination to offload HTTPS encryption and decryption from Bitbucket, and
  • cookie-based session affinity ("sticky sessions").

If you have no specific preference for your load balancer, Installing Bitbucket Data Center provides instructions for haproxy, a popular Open Source software load balancer. 

Which databases are supported by Bitbucket Data Center?

All the usual databases supported by Bitbucket Server (see Supported platforms) are also supported by Bitbucket Data Center, with one exception: MySQL (all versions) is not supported for Bitbucket Data Center at this time due to inherent deadlocks that can occur in this database engine at high load.  If you currently use MySQL, you should first migrate your data to another supported database such as PostgreSQL. See Connecting Bitbucket Server to an external database for more information. 

Can I migrate databases for a Bitbucket Data Center instance while clustered?

No. The Database Migration Wizard is not supported in Bitbucket Data Center instances while more than one cluster node is running. To migrate databases for a Bitbucket Data Center instance, you should perform the migration first, before starting multiple cluster nodes.

What methods are supported to backup a Bitbucket Data Center clustered instance?

Zero downtime backup is a technique introduced in Bitbucket 4.8 which backs up a Bitbucket instance without requiring it be locked for maintenance. It requires your shared home directory to be on a file system volume capable of atomic (block level) snapshots, and your database to be capable of either atomic backup or point in time recovery. Use of these technologies allows you to take backups as often as you need (e.g., hourly), without inconveniencing your users and build agents with frequent downtimes.

DIY Backup is fully supported in all versions of Bitbucket Data Center. It doesn't matter whether you point your DIY Backup script to the URL of the Bitbucket instance (i.e., the load balancer) or a specific cluster node, provided the machine where your DIY Backup runs has connectivity to that endpoint. All cluster nodes will drain their running database and filesystem operations, and display the "Performing Backup" page for as long as your DIY Backup script runs with Bitbucket under lock.  Most DIY Backup scripts that work with Bitbucket Server should work with little or no modification with Bitbucket Data Center. 

The Bitbucket Backup Client is not compatible with clustered Bitbucket Data Center instances, even if you switch back to a single node. There are currently no plans to allow the Bitbucket Backup Client to work on clustered instances of Bitbucket.

Be sure to read the article Data recovery and backups for details about the different methods of backing up Bitbucket Data Center.

Do I have to back up Elasticsearch data as part of my backup strategy?

No. If you have a remote Elasticsearch instance then search indexes are maintained in Elasticsearch's data directory, but you don't have to include this in your backup as it can be completely rebuilt if necessary from the data in your home directory and database. Note that for large instances, rebuilding the Elasticsearch index can potentially take several hours. Search functionality could be degraded while an Elasticsearch index is rebuilt. 

Smart Mirroring

What are the benefits of Smart Mirroring?

Smart Mirroring can greatly improve Git clone speeds for distributed teams working with large repositories. Large repositories that take hours to clone from a Bitbucket instance over the Internet from the other side of the world can take minutes when cloned from a local mirror on a fast network.

See Smart Mirroring for more information. 

What version of Bitbucket do I need to use Smart Mirroring?

4.2.0 or higher.

Do my mirrors need to be on the same version as my primary Bitbucket instance?

No. Your primary Bitbucket instance and all mirrors can run any version of Bitbucket, provided it is 4.2.0 or higher. You can upgrade the primary Bitbucket instance and mirrors any time you like, they do not all have to be upgraded at the same time.

Does my primary Bitbucket instance need to be a cluster to use Smart Mirroring?

No. You must have a Data Center license to use Smart Mirroring, but you don't actually have to have more than one cluster node in your primary Bitbucket instance.

Can I mirror repositories from non-Bitbucket servers using Smart Mirroring?

No. Try one of the several mirroring add-ons for Bitbucket listed on Atlassian Marketplace.

Can I mirror repositories from my account in Bitbucket Cloud with Smart Mirroring?

Yes. See Smart Mirroring for Bitbucket Cloud for details.

Does Smart Mirroring provide higher availability if my primary Bitbucket instance is down or unreachable?

No. Mirrors authenticate and authorize all users' credentials by delegating to the primary Bitbucket instance, so if the primary instance is down or unreachable any mirrors can't serve requests to users either. 

Can I cluster a mirror?

Not at this time. Please contact us

Can I point my continuous integration server (Bamboo, Jenkins, ...) at a mirror?

Yes you can. Note, though, that Bitbucket mirrors do not currently support native integration with Bamboo, Jenkins, or other continuous integration servers, so you need to configure the repository with the plain "Git" type (not "Bitbucket Server" or "Stash"), and use polling mode triggers instead of push notifications. Please contact us for more information about this.

Can I install add-ons on a mirror?

No.

Does Git Large File Support (LFS) work with Smart Mirroring?

Yes. From Bitbucket Data Center 4.5, when a file managed by LFS is first requested from a mirror it will be streamed to the user and cached locally for future requests. This strikes a balance of performance, where commonly requested files are cached for fast access, and storage efficiency, where historical files managed by LFS that are not of interest to mirror users are not actively synced.

How secure is Smart Mirroring?

The short answer to this is: No less secure than your primary Bitbucket instance. 

The slightly longer answer is: Smart Mirroring has been designed to require secure communication throughout and restrict all functionality to the appropriate privilege level (e.g., system administrator) to ensure the security of all your sensitive information. But mirrors must necessarily store copies of your repository data, so you must ensure that not only your primary Bitbucket instance but all your mirrors have been set up with the best security practices to prevent unauthorized access via some other route. Bitbucket prevents or warns about some common misconfigurations that are considered insecure. For example, if you attempt to set up a new mirror without SSL Bitbucket will not let you proceed, or if you configure Bitbucket's user account to allow all users read permission to repository data in the ${BITBUCKET_HOME} directory Bitbucket will give you a warning. But it does not catch every possible misconfiguration, and it is your responsibility to set up mirrors with the same stringent security practices as your primary Bitbucket instance.

Among the best practices recommended for both mirrors and standard Bitbucket installations are:

  • Ensure the primary instance and all mirrors have valid SSL certificates and do not allow requests over plain HTTP.
  • Lock down the mirror environments against unauthorized login or other access by anyone except a system administrator.
  • Never run the Bitbucket application as root, either on standard instances or mirrors.
  • Configure the Bitbucket user account (typically atlbitbucket) securely, for example by allowing login only with SSH keys instead of passwords and configuring a umask that doesn't allow other users to read and write files by default.
  • Make sure your operating system and all packages/components (e.g., Java, Git, proxies) are fully patched with the latest updates.
  • Avoid running other services that may be vulnerable on the same environment as your Bitbucket primary instance or mirrors.

Watch the Supported platforms page for minimum versions of third party software components and known versions to avoid. See the documentation for your vendor's operating system for more information about securing services in that environment. 

Smart Mirroring performance

How much faster is a mirror than my primary Bitbucket instance?

It depends. The speedup will vary with repository size, network bandwidth and latency, how well provisioned and heavily loaded your mirror is, and how fast the CPU is on the user's machine running git. For small repositories and fetches there may not be much in it. For moderately large repositories with good network connections the speedup will be a few times (or in absolute terms, measured in minutes).  For enormous (multi-GByte) repositories with not-so-good network connections the speedup will be considerably more, and in absolute terms could easily be measured in hours.

No, seriously, how much faster is a mirror?

We use local mirrors all the time, and we have some smallish repositories (100's of MBytes) that normally take 2 minutes to clone from the other side of the world take about 40 seconds from a local mirror (3x speedup). Larger repositories (> 4 GBytes) that take over an hour to clone from the other side of the world take 2 – 3 minutes from a local mirror (anywhere from 25x to 50x speedup).

How much CPU, memory, I/O, etc. does a mirror need?

Just as with your primary Bitbucket Data Center instance, your mirrors need to be provisioned with enough CPU, memory, and I/O resources to handle their peak workload. See Scaling Bitbucket Server for more information.

A mirror of course needs as much disk space for repository data as the space those repositories take up on primary Bitbucket instance. (Initially the mirror may use a little less space for repository data than the identical repositories on the primary instance, due to Git's garbage collection and repacking, but this difference will tend to stabilize with time.) As with standard Bitbucket instances, provisioning more disk space is better for performance, as Bitbucket uses some free disk space for its SCM cache. 

Will mirrors increase or decrease the load on my primary Bitbucket instance?

It depends. Mirrors take some load (possibly a lot) off their primary instance as every clone or fetch from a mirror represents an operation that the primary instance doesn't have to handle. On the other hand mirrors also add a bit of load to their primary instance as they receive webhook notifications informing them of pushes and other events on all the repositories they are mirroring, and periodically perform fetches from the upstream to keep those repositories in sync. This additional load is usually small, but if you have a lot of mirrors mirroring repositories that receive a lot of pushes and/or don't get cloned very much, it can add up. 

The time when mirrors generally add the most load to your primary instance is when they are first set up to start mirroring. This is because they start with no data and have to pull everything down from the primary instance from scratch. Bitbucket mirrors are designed to perform this initial sync of repository contents gradually so as not to place a lot of load on the primary instance all at once, but under some circumstances the added load can be significant – especially if your primary instance is already at or near capacity. To minimize the load and impact of mirrors on your primary instance, here's a few commonsense guidelines:

  • Know the load and capacity of your primary Bitbucket instance, and tune its SCM cache and scm-hosting ticket limits appropriately. See Scaling Bitbucket Server for Continuous Integration performance and Bitbucket Server is reaching resource limits for more information.
  • Provision a bit of spare capacity on your primary Bitbucket instance. The number of concurrent Git hosting operations sustainable by your instance is directly related to the amount of available CPU, memory, and IOPS resources. More free disk space will also help, as this is used by SCM cache to reduce load on other resources.
  • If possible, spin up new mirrors at periods of lighter than usual load (e.g., weekends or times when most users and build agents aren't active), not at periods of peak load. 
  • Avoid spinning up multiple new mirrors to mirror everything all at the same time. Spin them up one at a time, spaced some time apart. 
  • If some remote locations only need to work with some projects regularly, consider mirroring only the projects they need at that location, instead of all projects.
  • Once your mirrors are available, encourage your users to use them wherever possible so as to take as much load as possible off the primary instance. 

Do Smart Mirrors sync everything?

No. Smart Mirrors only synchronize the repositories in the Projects you specify. An option to sync all projects is available. Note that not all refs are synced to Smart Mirrors in real time. Certain refs, like those found under refs/pull-requests/, are only synced periodically. All public refs, like branches, found under refs/heads/, and tags, found under refs/tags/, are synced in real time.  

Smart Mirroring technical requirements

Can I use plain (unencrypted) HTTP for my mirror?

No. Both your primary Bitbucket instance and all your mirrors must use secure HTTP (HTTPS) and have valid SSL certificates issued by a Certificate Authority (CA). This is a strict requirement of Smart Mirroring on both the primary instance and all mirror(s), and cannot be bypassed. The mirror setup wizard will not let you proceed if either the mirror or the primary instance does not have a valid SSL certificate.  

Can I disable SSH when I use Smart Mirroring?

No. Mirrors use SSH to keep their repositories synchronized with the primary instance and cannot use HTTP or HTTPS for this. It is also not possible to disable SSH access to mirrors by users. See Enabling SSH access to Git repositories in Bitbucket Server for instructions on enabling SSH access on your primary instance.

Can I clone or fetch from a mirror using my SSH key or my HTTPS credentials?

Either. Or both. Mirrors allow clones and fetches over both SSH and HTTPS, and provide identical authentication mechanisms (SSH key, project/repository access key, HTTP credentials) and authorization policies (global permissions, project permissions, repository permissions, and/or other customizations) as the primary instance.

Do mirrors support CAPTCHA?

Yes. Mirrors don't actually do the CAPTCHA themselves, though, they delegate all this up to the primary Bitbucket instance. So it doesn't matter whether the failed authentication attempts are on the primary instance or a mirror, if you perform too many of them it will trigger CAPTCHA of the account on the primary instance, and you'll need to pass a CAPTCHA challenge to unlock it. 

What external databases are supported by Smart Mirroring?

There is no need to set up an external database in the mirror itself. The internal H2 database is sufficient.

The primary Bitbucket instance, of course, should use one of the supported external databases listed in Supported platforms

Does Smart Mirroring support Amazon Web Services (AWS)?

Yes.

What license do I need for my mirrors?

You don't need any license in the mirror itself. Your primary Bitbucket instance, though, must have a valid Data Center license for mirrors to work.

How do I back up my mirrors?

Mirrors do not really store any persistent state that can't be reconstructed by just setting it up again with the same primary Bitbucket Data Center instance from scratch. There is no need for elaborate backup strategies for mirrors. You should, of course, back up its important configuration files such as SSL certificate, server.xmlconfig/ssh-server-keys.pembitbucket.properties file, and so on in a safe place. But these configuration files typically do not change while the mirror is running, so ongoing backup is not required.

How do I upgrade a mirror node?

The same way as a standard Bitbucket Server instance, see the Bitbucket Server upgrade guide.

Bitbucket Data Center licensing

How do I purchase a license for Bitbucket Data Center?

Just go to https://www.atlassian.com/software/bitbucket/enterprise/data-center and click "Buy now". 

Can we move from our perpetual license to Bitbucket Data Center and then go back to Bitbucket Server later? Will there be any cost to change back?

While your subscription to Bitbucket Data Center is valid, you can easily run only one node if you want to. When your subscription expires, you can purchase a single server license to return to Bitbucket Server.

Do I have to buy licenses for add-ons on each node? 

No. Add-ons are sold at the Bitbucket Server rate, but the user tier has to exceed or match the Bitbucket Data Center license tier. For example, if you have 3,000 users in Bitbucket Data Center, then you would buy the add-on for 2,000-10,000 users.

To use add-ons with Bitbucket Data Center, simply install the add-on once as normal. You do not need to install on each cluster node individually.

Do you have an unlimited users license?

No. 

Can I continue to use developer licenses in my test environment?

Yes, you can use a developer license for a staging environment.

What happens if we stop paying for the Bitbucket Data Center subscription service?

You can still access your information in Bitbucket. However, you aren't able to push or make any other modifications in your instance. 

Troubleshooting Bitbucket Data Center clustering

I get "Lock file .../shared/.lock cannot be obtained in home directory." What's going on?

A common reason for this error is that your NFS lock service is disabled or misconfigured. Make sure the lockdportmap, and dbus services are all running on your NFS server, and restart Bitbucket. See  Installing Bitbucket Data Center for more information.

If you still have leftover .lock file(s) after shutting down all your instances of Bitbucket, it's safe to delete them and start over.

Does Bitbucket really lock the shared home directory?

Yes, Bitbucket creates .lock files in both ${BITBUCKET_HOME} and ${BITBUCKET_HOME}/shared.

Bitbucket Server acquires an exclusive lock on both directories, preventing any other instances of Bitbucket (Server or Data Center) from using them.

Bitbucket Data Center acquires an exclusive lock on ${BITBUCKET_HOME} (preventing any other Bitbucket instances from using it) and a shared lock on ${BITBUCKET_HOME}/shared (allowing it to be shared with other Bitbucket Data Center cluster nodes but not Bitbucket Server instances).

These checks might seem strict, but they are in place to make sure that data loss through accidentally misconfiguring Bitbucket instances to use the same home directory can never happen. 

My Bitbucket Data Center instance gets slow or unresponsive under heavy load, and in my logfiles I see a lot of "org.hibernate.exception.JDBCConnectionException: Could not open connection" or "java.sql.SQLException: Timed out waiting for a free available connection." What's going on?

These errors mean Bitbucket is running out of connections to the database, which could be due to a number of underlying causes:

  1. Your database may be misconfigured not to allow enough concurrent connections. Bitbucket by default uses up to 80 connections  per cluster node , which can exceed the default connection limit of some databases. For example, in PostgreSQL the default limit is usually 100 connections. If you use PostgreSQL, you may need to edit your  postgresql.conf  file, increase the value of  max_connections, and restart Postgres. See Installing Bitbucket Data Center for more information. 
  2. The system clocks on your cluster nodes may be drifting out of synchronization or may be being tampered with. This can sometimes be an issue in virtualized environments if time synchronization between the host OS and guest OS is misconfigured. Make sure all your cluster nodes are configured with the same timezone and are running the NTP service with identical configuration. See Installing Bitbucket Data Center for more information.  

If this problem persists, please contact Atlassian Support.

My Bitbucket Data Center instance gets slow or unresponsive for no apparent reason, and in my logfiles I see a lot of "com.hazelcast.core.OperationTimeoutException: No response for 60000 ms. Aborting invocation!" or "c.a.s.i.s.s.DefaultPublicKeyAuthenticator Timed out while authenticating SSH user at ...". What's going on?

These errors mean that one or more Bitbucket cluster nodes can't talk to each other over the network, which could be due to a number of underlying causes:

  1. Network failures. Check that all your cluster nodes can ping each other without too much latency or packet loss, and that traffic to and from port 5701 is allowed by your router and firewall configuration. 
  2. The Java virtual machines running Bitbucket may be running out of memory. Try following the guidelines in Scaling Bitbucket Server and How to debug Out of Memory Heap Space.
  3. The system clocks on your cluster nodes may be drifting out of synchronization or may be being tampered with. This can sometimes be an issue in virtualized environments if time synchronization between the host OS and guest OS is misconfigured. Make sure all your cluster nodes are configured with the same timezone and are running the NTP service with identical configuration. See  Installing Bitbucket Data Center  for more information.   

If this problem persists, please contact Atlassian Support.

Last modified on Aug 13, 2017

Was this helpful?

Yes
No
Provide feedback about this article

Not finding the help you need?

Ask the community

Powered by Confluence and Scroll Viewport.