SSH Operations fail after upgrading Bitbucket Server/Datacenter
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
SSH operations fail after upgrading Bitbucket to 7.21.y. For example, git clone over SSH fails with the below error
git clone <SSH_CLONE_URL>
Cloning into 'repository'...
Connection reset by 10.138.15.76 port 7999
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Environment
Bitbucket 7.21.18, 7.21.16, 8.9.7
Java JDK version < Java 8u251 (or)
Java 11 < 11.0.5
Diagnosis
Enable SSH debug logging on Bitbucket server and replicate the issue by cloning the repository over SSH
curl -u <admin_user>:<admin_user_password> -v -X PUT -d "" -H "Content-Type: application/json" <BITBUCKET_BASE_URL>/rest/api/latest/logs/logger/org.apache.sshd/debugThe below log messages are seen in Bitbucket server log (atlasssian-bitbucket.log)
<DATE_TIMESTAMP>,397 DEBUG [sshd-DrainableSshServer[xxxxx](port=7999)-nio2-thread-1] o.a.sshd.common.io.nio2.Nio2Session handleReadCycleFailure(Nio2Session[local=/XX.XX.XX.XX:7999, remote=/10.133.0.31:51310]) NoSuchProviderException after 5950866 nanos at read cycle=2: JCE cannot authenticate the provider BC
<DATE_TIMESTAMP>,397 DEBUG [sshd-DrainableSshServer[xxxxx](port=7999)-nio2-thread-1] o.a.sshd.common.io.nio2.Nio2Session exceptionCaught(Nio2Session[local=/XX.XX.XX.XX:7999, remote=/10.133.0.31:51310]) caught NoSuchProviderException[JCE cannot authenticate the provider BC] - calling handler
<DATE_TIMESTAMP>,397 DEBUG [sshd-DrainableSshServer[xxxxx](port=7999)-nio2-thread-1] o.a.s.c.s.h.SessionTimeoutListener sessionException(ServerSessionImpl[null@/XX.XX.XX.XX:<PORT>]) NoSuchProviderException: JCE cannot authenticate the provider BC
To disable SSH debug logging
curl -u <admin_user>:<admin_user_password> -v -X PUT -d "" -H "Content-Type: application/json" <BITBUCKET_BASE_URL>/rest/api/latest/logs/logger/org.apache.sshd/warnCause
Bitbucket started with an older JDK8 version JDK8-73. The issue is caused due to JDK bug JDK-8214096 which is fixed in higher JDK versions.
Solution
Upgrade JDK to the fixed version and start Bitbucket server with the fixed JDK version.
Fixed versions as per JDK-8214096
| Fixed Version | |
|---|---|
| Java 8 | >=JDK8u251b04 >=JDK8u261b01 |
| Java 11 | 11.0.5 b01 |