Reduce repository size

Bitbucket is great for keeping your source files and managing changes, especially when working in a team, but it's not the best place to keep large binary files, such as executables and other build artifacts, or videos and other media files.

We recommend repositories be kept under 1.0 GB to help ensure that our servers are fast and downloads are quick for our users. Bitbucket Cloud repositories have a 2.0 GB limit. When that limit is reached, you will only be able to push changes that undo the latest commits. There is also a hard limit of 5.0 GB. When that limit is reached, you will not be able to push any changes, and your repository becomes read-only.

If you need to keep large files in Bitbucket, consider introducing Git Large File Storage (Git LFS) as part of your workflow.

Repository size limits

If the size is over these limits you'll see a warning in the Repository details panel:

Over 1.0 GB limit

You’ll get a warning on the command line if you push commits while over the 1.0 GB limit:

$ git push 
Enumerating objects: 5, done. 
Counting objects: 100% (5/5), done. 
Delta compression using up to 12 threads 
Compressing objects: 100% (2/2), done. 
Writing objects: 100% (3/3), 332 bytes | 332.00 KiB/s, done. 
Total 3 (delta 0), reused 0 (delta 0) 
remote: This repository is currently 1.0 GB in size. If it exceeds 2 GB it will be put into read-only mode. 
remote: Learn how to reduce your repository size: 
	3db4505..0393b72 master -> master 
$ _

If you see this warning you should perform maintenance on the repository to remove large files; see Removing large files below.

Over 2.0 GB limit

If the size exceeds the 2.0 GB limit you will not be able to push any more commits:

$ git push
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 12 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 406 bytes | 406.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: b'Repository is over the size limit (2 GB) and will not accept further additions.
remote: Learn how to reduce your repository size:
remote: '
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to ''
$ _

To continue making changes you’ll need to undo the last commit; see Undoing the last push below. This will bring the size below the 2.0 GB limit and remove the push restriction, allowing you to perform maintenance on the repository.

Over 5.0 GB limit

Above 5.0 GB all changes are rejected. You will need to raise a support request to get help reducing the size of your repository.

Undoing the last push

To remove large files you need to rewrite history; otherwise, Git just keeps the large files in the history.

Remove the push restriction to rewrite history.

Rewind the branch containing the bad commit to just before that commit, and then prune the last commit to bring the repository size under the repository size limit. This process is assuming that the bad commit is only on one branch and hasn’t been merged to other branches.

It is essential to inform anyone else using the affected branch that you are undoing the last commit, and not to merge that commit into any other branches.

Remote history in Bitbucket looks something like the following:

o---o---o---o <= last commit in Bitbucket

Local history might look similar to the following:

o---o---o---o---o <= last local commit failed to push
        reset to here and push

Rewind the remote branch

On the branch containing the last commit:

  1. Create a temporary branch to keep any local commits.

  2. Reset the original branch to the commit just before the last one pushed (or before the bad commit containing the large files, if multiple commits were pushed).

  3. Push the new head to Bitbucket.

  4. Restore local changes – you won’t be able to push these yet, you’ll need to remove any large files first.

git branch <keeper>
git reset --soft @{u}^
git push --force
git merge --ff-only <keeper>
git branch -d <keeper>

Using the --soft option will keep any commits you haven't pushed yet and any changes you haven’t committed, so you can push them later once you’ve removed any large files. If you don’t have changes you want to keep, you can use the --hard option and skip Steps 1, 4, and 5 in the command line example above.

Remove dangling commit

Now you'll need to raise a support request to prune the last commit and run garbage collection so that the last commit is completely removed and the repository’s size is reduced.

Remote history in Bitbucket now looks something like the following:

            prune this

Once that’s done pushes will work again, so you can go ahead and reduce the repository size to below the 1.0 GB limit by removing large files; see Removing large files below.

Removing large files

Once pushes are unblocked, you can go ahead and remove large files from the repository. Below are some resources to help you maintain your Git repository and provide more information about using Git LFS.

If you want to keep a lot of large files without paying for extra LFS storage you’ll need to put them elsewhere; see Options for storing large files below for a few of the available options.

Once large files have been removed, it is a best practice to clone the repository from Bitbucket; otherwise, if someone does a force push, they will push the large files again and you’ll be back to where you started.

Avoiding large commits

There are a couple of things you can do to avoid accidentally adding large files to your repository:

  • Tell Git to ignore the kinds of files you don’t want to include.

  • Install automation to prevent large commits from being created.

Ignore large files

You can tell Git to exclude files from commits by adding pathname patterns to a file called .gitignore in the directory containing your local repository. 

For example:

# File types to ignore
# Directories to ignore

In general, you may want to tell Git to ignore:

  • build artifacts – best to put all these in a directory, for example Maven puts them in a target directory.

  • IDE settings – you don’t usually want these in the repository, for example ignore the .idea directory.

  • dependencies – exclude caches of dependencies, for example Python’s virtualenv or node’s local packages.

  • media files – a git repository is not the best place to keep large audio or video files.

Block large commits

To prevent any large files getting included in commits, you can install a local hook that checks the size of files in every commit, therefore rejecting the commit if it is too large.

Start with the check_added_large_files hook, by copying it into the repository so anyone can install it by adding it as the pre-commit hook for their local repository:

ln -s .git/hooks/

You can change the copy of the script in your repository to work however you want, and everyone will get the latest logic when they pull.

Options for storing large files

Your Bitbucket repository is the best place to keep source files. There are better places to keep other files generated from that source. Here we give some examples but encourage you to explore all the available options.

Use an artifact repository

There are many services that store build artifacts, two popular examples being:

Configure your build process to upload build artifacts to these repositories so they can be shared. Also, make sure .gitignore is configured to exclude build artifacts from commits.

Use a Docker repository

If your Bitbucket repository is used to build executables, consider building them as Docker images.

Your build process can push the images to a Docker repository hosted on Docker Hub, or a local Docker repository.

Use AWS S3

Rather than storing large media files in your Bitbucket repository, upload them to an S3 bucket where they can be easily downloaded.

Use Git LFS

If the files really need to be part of the Bitbucket repository, use the Large File Storage available on your plan. You can buy more storage if required.

Tell Git to use LFS for specific types of file using a wildcard pattern:

git lfs track "<pattern>"

For example, to use LFS for MP4 movies:

git lfs track "*.mp4"

*The quotes in the example above are important.

Last modified on Nov 27, 2019

Was this helpful?

Provide feedback about this article
Powered by Confluence and Scroll Viewport.