Summary

Bitbucket Data Center has default rules to test against a secret that is mistakenly included in the files committed to Bitbucket.

Other than that, Bitbucket also allow adding new rule and give it a custom name and the new rule at the Secret Scanning feature at the project or repository level. 

In this article, we share some guidance on how to test these secret scanning rules when there is a need for testing and validating whether the custom user-created rule or default Bitbucket secret scanning rule works.

Environment

Bitbucket Data Center 8.3+

Diagnosis

The user adds a secret such as below into the code but no notifications are sent even though there is a secret in the code line.

Example : $headers = @{'Authorization': 'Basic c3ZjLUlOTkgxNDUxMTA6QW9uT25lRGV2b3BzQDEyMw==';'x-atlassian-token'='no-check'} 

Solution

Bitbucket uses RE2J for the regex rule for secret scanning.

Below are some useful external tools that you could use for testing:-

• Regex tester for Golang
• Google sheets regex functions
  
  

There is a difference between Bitbucket scanner and regex tool. Using the regex tool above, it tries to find the occurrence of a regex pattern within a given string. However, in certain version of Bitbucket it tries to match the whole string in a line against the custom/default pattern in a line which means the line should not contain any other pre or post-characters that wraps the secret in the line for it to pass. 

Email notifications are sent out to everyone involved in the commit history of the secret: the authors, committers, and the developer who pushed or merged the code containing secrets into the repositories. Therefore, ensure that mail notifications feature is setup successfully for the user who is testing out secret scanning.

For example a sample below on Git Personal Access Token rule, matches the default Bitbucket rule \b(?i)glpat-[0-9a-z\-]{20}\b and email notifications will be sent out. If you would like to receive an email immediately, you will need to change the email notifications to be sent immediately in the user profile.

$gitLabAPIKey="glpat-XFZAzGLyz8Jjy1nw4XPB" 

If we use the tool Regex tester for Golang, we can place the Test String as $gitLabAPIKey="glpat-XFZAzGLyz8Jjy1nw4XPB" and Regular Expression as \b(?i)glpat-[0-9a-z\-]{20}\b

The tool will show that test string and pattern matches and the same for Bitbucket, as the whole string matches the rule pattern.

Resolution to the diagnosis case

On detecting Basic Authentication secret in code, the default rule in Bitbucket is the pattern (\"|')?Basic [A-Za-z0-9\\+=].
It will not match below test string, because the test string contains a pre and post characters that wraps the secret.

Example : $headers = @{'Authorization': 'Basic c3ZjLUlOTkgxNDUxMTA6QW9uT25lRGV2b3BzQDEyMw==';'x-atlassian-token'='no-check'} 

The above will work if the Line Pattern rule is tweaked to .*(\"|')?Basic ([A-Za-z0-9\\+=]{44}(\"|'))?.* by adding .* to the pre and post of the initial Line pattern so that it will match the test string above.

Changes in newer Bitbucket version
This ticket BSERV-14161 - Getting issue details... STATUS addresses the above diagnosis and in newer version of Bitbucket, the secret scanning feature will match the secret found in any part of an entire code line.