Purpose

Developers often mistakenly commit company passwords or secrets into their code and it becomes necessary to audit and remove them. This article will cover the best ways to prevent and remove them.

  BSERV-11064: Sensitive data removal and tracking for Bitbucket is a feature request to make this process easier. Please watch and vote on it for updates.

Solution

Preventative

The most effective way to ensure passwords aren't stored in Git repositories is to prevent them from being pushed in the first place. Git doesn't make it easy to remove passwords from every commit and branch once a password is in a file.

PRE-RECEIVE HOOK

A pre-receive hook that prevents any user from pushing code that contains a certain set of passwords is a great measure to take to prevent passwords from being pushed.

It's likely that you can write your own hook with similar instructions to Pre-receive Hook Plugin Module. Please see these as well:

• Using repository hooks
• How to create a simple hook in Bitbucket Server

There are also some third party plugins that could assist with these kinds of requirements as well like Script Runner's solutions.

 Please note that a script like this would have a performance impact on your instance, especially if not implemented properly.

TIPS FOR YOUR DEVELOPERS TO PREVENT PLAIN TEXT PASSWORDS

There are a few simple tricks to avoid committing certain strings from the client side as well. It's a great idea to train your developers with the following tips:

• Use environmental variables instead of setting local variables. This will let your software pull the passwords/tokens from the system and thus keeping your code clean.
• Use separate files that wont be committed to store your secrets and passwords.
• Use a visual Git program (like SourceTree). Visual programs generally make it easier to see exactly which files will be added, deleted, and modified with each commit.
• Avoid the catch-all commands git add . and git commit -a on the command line—use git add filename and git rm filename to individually stage files, instead.
• Use git add --interactive to individually review and stage changes within each file.
• Use git diff --cached to review the changes that you have staged for commit. This is the exact diff that git commit will produce as long as you don't use the -a flag. 

• You could have your developers install something like Git Secrets that would prevent them from committing based on regex patterns.

Reactive Approach

While a reactive approach is not ideal, we understand sometimes secrets slip into to your repository. This is not easy to fix but it is possible.

Scanning for Passwords

The best approach to identify which repositories contain passwords is to implement something at the file system level. 

Bitbucket Server stores all of its repositories in $BITBUCKET_HOME/shared/data/repositories. There you will find all of Bitbucket's bare repositories listed numerically by repository ID. Repository ID which can be associated with the repository name by using the steps in Identifying a Repository on the Bitbucket Server. The easiest way to find passwords is to iterate through each bare repository directory with a git command that will search each branch for changes that match a string or regex pattern.  Here are a couple examples.

The following example will output each commit containing a change with the associated line. This command also works with Regex patterns.

 git grep 'my_secret_password' $(git rev-list --all)

Alternatively you can use the following command: 

git log -p --all -S 'my_secret_password'

This option is more verbose and also provides you the commit author with more context. 

 Note that you will need to replace "my_secret_password" with your own string to search for. 

The above workflow could be automated to send an email when a suspect file is found. You could also build a plugin that displays a message through the Bitbucket UI. The specifics of what you implement are entirely up to you and your ability to code them.

The advantages here are that you offload the work from Bitbucket so there is little performance impact to the instance.

Removing Passwords

Once you have compiled a list of files with passwords, you will need to manually remove that string from Git history which can be tricky. We recommend that you use in-built git functions to search and sanitise your repositories that currently contain sensitive information. This helpful article Git How-To: Remove Your Password from a Repository should put you on the right track. Alternatively, you might find something like Gitleaks helpful for scanning and detecting sensitive information in your repository.

Another alternative to git filter-branch is BFG Repo-Cleaner which also covers removing passwords, credentials & other private data.

 As a side note, Premier Support does not support custom plugin development, however, if you'd like you can contact an Atlassian Partner. Our partners may be able to build a custom plugin for you that has this functionality. Please note that this is a separate paid service.