Missing Commits in Bitbucket Server
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Problem
You are not seeing previously available commits in your repository. These missing commits are usually a result of a force push that has rewritten history in the Git repository.
Another, less common, cause for missing commits is after a migration to a new filesystem using rsync without the --delete option (see Missing commits in Bitbucket after a filesystem migration for details).
Force pushing and rewriting history is native to Git, therefore Atlassian Support cannot guarantee that we can provide support to help recover missing commits after this type of action.
We recommend establishing Branch Permissions to prevent rewriting history and protecting important branches, see Using branch permissions for more information.
Diagnosis
Gather all ref changes
The first step to recovering from a force push is to identify the commit hashes that changed. Typically this means that a branch pointer was moved to a previous commit in time which leaves all children of that commit as dangling commits and will effectively show as missing in the Bitbucket Server UI. The following SQL query can be used to identify all ref changes for a branch. Changes to refs are normal operations so many of the results returned will be expected.
select p.project_key, r.slug, pr.ref_id, pr.change_type, pr.from_hash, pr.to_hash, nu.name
from sta_repo_push_ref pr
join sta_repo_activity ra on ra.activity_id = pr.activity_id
join repository r on r.id = ra.repository_id
join project p on p.id = r.project_id
join sta_activity a on a.id = pr.activity_id
join sta_normal_user nu on nu.user_id = a.user_id
where p.project_key = '[PROJECT KEY]'
and r.slug = '[REPO SLUG]'
and pr.ref_id = 'refs/heads/[BRANCH_NAME]'
order by a.created_timestamp desc;The project_key should be uppercase. For example, the following query can be used to identify all ref changes on from the scratch repository in the PROJ project on the master branch:
select p.project_key, r.slug, pr.ref_id, pr.change_type, pr.from_hash, pr.to_hash, nu.name
from sta_repo_push_ref pr
join sta_repo_activity ra on ra.activity_id = pr.activity_id
join repository r on r.id = ra.repository_id
join project p on p.id = r.project_id
join sta_activity a on a.id = pr.activity_id
join sta_normal_user nu on nu.user_id = a.user_id
where p.project_key = 'PROJ'
and r.slug = 'scratch'
and pr.ref_id = 'refs/heads/master'
order by a.created_timestamp desc;This returns the following rows (truncated to the top 3):
project_key slug ref_id change_type from_hash to_hash name
PROJ scratch refs/heads/master 3 d63f3332e507e1e7939852a906d49809d47e9847 e168c777948c36f81ee7c9e3515bc0635f4cfec3 user
PROJ scratch refs/heads/master 3 b66a493fc92bd1a8337e9b5566563f4915bb44db d63f3332e507e1e7939852a906d49809d47e9847 user
PROJ scratch refs/heads/master 3 c783a47d91646ea924d5347412f809da92ff6bc4 b66a493fc92bd1a8337e9b5566563f4915bb44db userIdentify the ref change that removed commits
Using the /rest/api/latest/projects/PROJ/repos/REPO/commits?until=FROM_HASH&since=TO_HASH for each row above, we can identify when the force push occurred. This endpoint asks Bitbucket Server "Which commits are reachable from FROM_HASH that are not on TO_HASH?" During normal operations, no commits will be returned, but if commits are returned, it indicates that a force push has occurred.
From the example above, the following REST calls would be run:
curl -u jeff -H "Accept: application/json" "https://bitbucket.company.com/rest/api/latest/projects/PROJ/repos/scratch/commits?until=d63f3332e507e1e7939852a906d49809d47e9847&since=e168c777948c36f81ee7c9e3515bc0635f4cfec3"
{"values":[],"size":0,"isLastPage":true,"start":0,"limit":25,"nextPageStart":null}curl -u jeff -H "Accept: application/json" "https://bitbucket.company.com/rest/api/latest/projects/PROJ/repos/scratch/commits?until=b66a493fc92bd1a8337e9b5566563f4915bb44db&since=d63f3332e507e1e7939852a906d49809d47e9847"
{
"values":[
{
"id":"b66a493fc92bd1a8337e9b5566563f4915bb44db",
"displayId":"b66a493fc92",
"author":
{
"name":"user",
"emailAddress":"user@company.com",
"id":51,
"displayName":"Jeff",
"active":true,
"slug":"user",
"type":"NORMAL",
"links":
{
"self":[
{
"href":"https://bitbucket.company.com/users/user"
}]
}
},
"authorTimestamp":1462808100000,
"message":"commit on master",
"parents":[
{
"id":"c783a47d91646ea924d5347412f809da92ff6bc4",
"displayId":"c783a47d916"
}]
},
{
"id":"c783a47d91646ea924d5347412f809da92ff6bc4",
"displayId":"c783a47d916",
"author":
{
"name":"user",
"emailAddress":"user@company.com",
"id":51,
"displayName":"Jeff",
"active":true,
"slug":"user",
"type":"NORMAL",
"links":
{
"self":[
{
"href":"https://bitbucket.company.com/users/user"
}]
}
},
"authorTimestamp":1459809480000,
"message":"test",
"parents":[
{
"id":"d63f3332e507e1e7939852a906d49809d47e9847",
"displayId":"d63f3332e50"
}]
}],
"size":2,
"isLastPage":true,
"start":0,
"limit":25,
"nextPageStart":null
}curl -u jeff -H "Accept: application/json" "https://bitbucket.company.com/rest/api/latest/projects/PROJ/repos/scratch/commits?until=c783a47d91646ea924d5347412f809da92ff6bc4&since=b66a493fc92bd1a8337e9b5566563f4915bb44db"
{"values":[],"size":0,"isLastPage":true,"start":0,"limit":25,"nextPageStart":null}The second cURL command shows that the push to master which moved the pointer from b66a493fc92bd1a8337e9b5566563f4915bb44db to d63f3332e507e1e7939852a906d49809d47e9847 was a force push and left two dangling commits.
Cause
A force push (push -f) was used to update the repository, altering the repository history.
Resolution
Before making any changes to the repository, create a copy of the folder that stores the bare repository on the server.
The location of this folder can be found in Bitbucket Server's web interface, on the Repository Settings page, next to: Location on disk.
The following steps need to be performed directly on the server where the missing commits need to be recovered.
Clone the bare repository to a temporary location to work with
git clone /var/atlassian/application-data/bitbucket/shared/data/repositories/<REPO_ID> temp-cloneCheckout the branch that the force push was performed on
git checkout <BRANCH>Merge the
from_hashinto the current tip of the branch, which essentially restores all of the commits that were removed while retaining any new commitsgit merge <FROM_HASH>- Resolve any conflicts and commit the merge
Add the Bitbucket Server repository as a local remote
git remote add recovery https://bitbucket.company.com/scm/<PROJ>/<REPO>.gitPush the new changes to Bitbucket Server
git push recovery <BRANCH>