Git operations fail due to "git-pack-objects died with error"

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Problem

Git commands don't complete successfully.

The following appears in the command console output:

error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500 Internal Server Error

The following appears in the atlassian-bitbucket.log

ERROR [http-nio-7990-exec-52] <username> @1WU14K3x805x122891x1 193.39.66.214 "POST /scm/<project_key>/<repository_slug>.git/git-upload-pack HTTP/1.1" c.a.s.i.s.g.p.h.GitSmartExitHandler <project_key>/<repository_slug>[10]: Read request from 193.39.66.214 failed: com.atlassian.utils.process.ProcessException: Non-zero exit code: 1
The following was written to stderr:
error: git upload-pack: git-pack-objects died with error.
fatal: git upload-pack: aborting due to possible repository corruption on the remote side.

Cause

There may be several causes for this problem:

Cause #1 - Corrupted Git repository due to filesystem corrupton

The repository has become corrupted as a consequence of a file system corruption.

Note: cloning the repository directly on the file system (without using Bitbucket Data Center) may still succeed since a Git clone does not run any additional checks and assumes that the repository is in a consistent state.


Cause #2 - Corrupted Git repository due to insufficient memory

The repository has become corrupted after a repack operation failed due to insufficient memory. Attempting a manual repack of the repository (see Workaround for Cause #1) shows the following error.

Delta compression using up to 4 threads
error: pack-objects died of signal 9)


Workaround

If the steps below don't resolve the problem, the repository may be not recoverable. In this case, the situation can be resolved by restoring a backup generated prior to the corruption.

Cause #1

Manually repack the repository.

  • Stop Bitbucket Server
  • Copy the $BITBUCKET_HOME/shared/data/repositories/<repository_id> folder. This is to make sure that a backup of the entire repository is available.
  • Run the following commands from the $BITBUCKET_HOME/shared/data/repositories/<repository_id> path:

    git fsck --no-dangling
    git repack -adfln --keep-unreachable --depth=20 --window=200
  • Start Bitbucket Server


Cause #2

Set a limit for the memory allocated when running the repack.

  • Stop Bitbucket Server
  • Copy the $BITBUCKET_HOME/shared/data/repositories/<repository_id> folder. This is to make sure that a backup of the entire repository is available.
  • Navigate to the $BITBUCKET_HOME/shared/data/repositories/<repository_id> path.

  • Add --window-memory=<n> to the git repack command and see if adjusting the value produces a different output and allows the repack to complete successfully.
git fsck --no-dangling 
git repack -adfln --keep-unreachable --depth=20 --window=200 --window-memory=500m
..
git repack -adfln --keep-unreachable --depth=20 --window=200 --window-memory=1g
  • Start Bitbucket Server


DescriptionThis article discusses about causes and workaround when git commands don't complete successfully.

Last modified on Feb 19, 2024

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.