Git Operations Fail - Permission denied - publickey
Problem
Having enabled SSH access in Bitbucket Server (and having added SSH keys to the correct user profile in Bitbucket Server) git operations fail with the following message on the console:
[drohan@test_box]$ git pull
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
Diagnosis
Enable debug logging for git to see the command that is failing (NOTE: curl isn't used for the SSH protocol):
export GIT_TRACE_PACKET=1
export GIT_TRACE=1
Sample output:
[drohan@test_box]$ git pull
trace: exec: 'git-pull'
trace: run_command: 'git-pull'
trace: built-in: git 'rev-parse' '--git-dir'
trace: built-in: git 'rev-parse' '--is-bare-repository'
trace: built-in: git 'rev-parse' '--show-toplevel'
trace: built-in: git 'ls-files' '-u'
trace: built-in: git 'symbolic-ref' '-q' 'HEAD'
trace: built-in: git 'config' '--bool' 'branch.master.rebase'
trace: built-in: git 'config' '--bool' 'pull.rebase'
trace: built-in: git 'rev-parse' 'q' '-verify' 'HEAD'
trace: built-in: git 'fetch' '--update-head-ok'
trace: run_command: 'ssh' '-p' '7999' 'drohan@kidney' 'git-upload-pack '\''/repo.git'\'''
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
Enable verbose logging for SSH (-v Verbose mode
) to identify why the error is happening:
% ssh -p 7999 -vT git@localhost
OpenSSH_5.9p1 Debian-5ubuntu1.4, OpenSSL 1.0.1 14 Mar 2012
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to localhost [127.0.0.1] port 7999.
debug1: Connection established.
debug1: identity file /home/josh/.ssh/id_rsa type 1
debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048
debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048
debug1: identity file /home/josh/.ssh/id_rsa-cert type -1
debug1: identity file /home/josh/.ssh/id_dsa type -1
debug1: identity file /home/josh/.ssh/id_dsa-cert type -1
debug1: identity file /home/josh/.ssh/id_ecdsa type -1
debug1: identity file /home/josh/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 2.0, remote software version SSHD-CORE-0.13.0
debug1: no match: SSHD-CORE-0.13.0
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1.4
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: RSA e8:6d:e4:fd:47:54:fe:12:d7:a1:85:43:fd:e0:cb:c5
debug1: Host '[localhost]:7999' is known and matches the RSA host key.
debug1: Found key in /home/josh/.ssh/known_hosts:2
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/josh/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
Agent admitted failure to sign using the key.
debug1: Trying private key: /home/josh/.ssh/id_dsa
debug1: Trying private key: /home/josh/.ssh/id_ecdsa
debug1: No more authentication methods to try.
Permission denied (publickey).
Cause
There are several causes (examples will be added as they become available):
- In the example above, the wrong private keys were being referenced:
/home/josh/.ssh/id_dsa
and/home/josh/.ssh/id_ecdsa
Resolution
- Verify that identity file you generated is working:
Verify the correct private key is works by explicitly sourcing it:
ssh -p 7999 -i ~/.ssh/id_rsa -vT git@localhost
If the command above is not working, please make sure you add the corresponding public key to Bitbucket server either as an access or personal SSH key. Copy and paste the content of the file below:
cat < ~/.ssh/id_rsa.pub
More details of the procedure below can be found on:
- SSH access keys for system use or
- SSH user keys for personal use
If the test above works, set the correct default SSH key location. The correct location for the source of the private key can be redefined:
% ssh-agent -l % ssh-add ~/.ssh/id_rsa