Problem

Bitbucket Server Backup Client fails, and the following is written in the backup logs (in debug mode):

2015-07-24 16:50:24,983 DEBUG [threadpool2] c.a.s.i.b.client.wink.LoggingHandler HTTP GET https://<BITBUCKET_SERVER_NAME>/stash/mvc/maintenance, headers: [Accept:[[application/json]], User-Agent:[[Wink Client v1.1.2]], X-Atlassian-Maintenance-Token:[[8eb296bb575a6dd2879c7b33adf795cec0cf085f]]]
2015-07-24 16:50:25,093 DEBUG [main] c.a.s.i.b.c.FixedRateProgressPublisher Progress publishing has stopped
2015-07-24 16:50:25,101 DEBUG [main] c.a.s.i.b.client.RestStashService Unhandled client exception encountered while Stash was performing a database backup
org.apache.wink.client.ClientRuntimeException: java.lang.RuntimeException: java.net.SocketException: Connection reset

Cause

Bitbucket Server uses a reverse proxy or load balancer (referred to from now on as proxy) in front of it (could be Apache, nginx, F5, or any other vendor). Proxy is also configured to check if the site is running periodically and it is blocking traffic to the server returns anything other than status 200. The backup client fails because it is not be able to check the status of the backup.

Workaround

Provide a connector that is used only by the backup client (and possibly Application Links) that bypasses the proxy. More details on setting up a connector to bypass the proxy here: Data Center How to bypass a reverse proxy or SSL in Application Links

When setting up a proxy bypass the ports should not be published to the users. If the backup is running on the Bitbucket Server / Stash server the port configured from external use can be blocked with firewall or IP table rules.

Resolution

Modify the proxy health check from hitting http(s)://BitbucketServer:Port/Context/Projects to check http(s)://BitbucketServer:Port/context/status instead.

• • Allow traffic to continue through to Tomcat when the json response is starting up, running, or maintenance and block the traffic to Tomcat when the status is shutting down or stopped.

The specific steps depend on the reverse proxy or load balancer being used. Check with that applications documentation for the specific steps that need to be taken in order to modify the endpoint that it is checking and which responses initiate a connection block and which responses are allowed to proceed.

Example Query Using curl:

curl http://localhost:7990/bitbucket/status

Example Responses:

Starting

{"state":"STARTING"}

Running:

{"state":"RUNNING"}

Maintenance:

{"state":"MAINTENANCE"}

Stopping

{"state":"STOPPING"}

Tomcat not Running:

Failed to connect to localhost port 7990: Connection refused