Bitbucket Server Backup Client fails with java.net.SocketException: Connection reset when using a proxy

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Problem

Bitbucket Server Backup Client fails, and the following is written in the backup logs (in debug mode):

2015-07-24 16:50:24,983 DEBUG [threadpool2] c.a.s.i.b.client.wink.LoggingHandler HTTP GET https://<BITBUCKET_SERVER_NAME>/stash/mvc/maintenance, headers: [Accept:[[application/json]], User-Agent:[[Wink Client v1.1.2]], X-Atlassian-Maintenance-Token:[[8eb296bb575a6dd2879c7b33adf795cec0cf085f]]]
2015-07-24 16:50:25,093 DEBUG [main] c.a.s.i.b.c.FixedRateProgressPublisher Progress publishing has stopped
2015-07-24 16:50:25,101 DEBUG [main] c.a.s.i.b.client.RestStashService Unhandled client exception encountered while Stash was performing a database backup
org.apache.wink.client.ClientRuntimeException: java.lang.RuntimeException: java.net.SocketException: Connection reset

Cause

Bitbucket Server uses a reverse proxy or load balancer (referred to from now on as proxy) in front of it (could be Apache, nginx, F5, or any other vendor). Proxy is also configured to check if the site is running periodically and it is blocking traffic to the server returns anything other than status 200. The backup client fails because it is not be able to check the status of the backup.

Workaround

Provide a connector that is used only by the backup client (and possibly Application Links) that bypasses the proxy. More details on setting up a connector to bypass the proxy here: How to bypass a reverse proxy or SSL in Application Links

When setting up a proxy bypass the ports should not be published to the users. If the backup is running on the Bitbucket Server / Stash server the port configured from external use can be blocked with firewall or IP table rules.

Resolution

Modify the proxy health check from hitting http(s)://BitbucketServer:Port/Context/Projects to check http(s)://BitbucketServer:Port/context/status instead.

    • Allow traffic to continue through to Tomcat when the json response is starting up, running, or maintenance and block the traffic to Tomcat when the status is shutting down or stopped.

The specific steps depend on the reverse proxy or load balancer being used. Check with that applications documentation for the specific steps that need to be taken in order to modify the endpoint that it is checking and which responses initiate a connection block and which responses are allowed to proceed.

Example Query Using curl:

curl http://localhost:7990/bitbucket/status

Example Responses:

Starting

{"state":"STARTING"}

Running:

{"state":"RUNNING"}

Maintenance:

{"state":"MAINTENANCE"}

Stopping

{"state":"STOPPING"}

Tomcat not Running:

Failed to connect to localhost port 7990: Connection refused 

Last modified on Apr 19, 2016

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.