Skip to end of metadata
Go to start of metadata

There may be times when you want to prevent a failing test from causing the whole build to fail.

Possible scenarios where this may be useful include:

  • You want to build an artifact despite there being a failing test, but can't do this while the plan build is failing.
  • In test-driven development (TDD), a test will fail until the functionality is implemented - you want to quarantine all but the relevant tests.
  • A test may give unpredictable results, perhaps because of infrastructure issues or dependencies.
  • You want to remove a test from a build, but don't want to alter or delete the test source code because doing so could affect another Bamboo plan.

In Bamboo, you can temporarily disconnect any test's results from the plan build results by quarantining the test. The test is still run whenever the plan is built, but the test's results do not affect the plan's build results.

You can always restore a test's results to the build results when required, for example if the test is now passing.

All the quarantined tests for a plan are displayed on the Quarantined Tests tab of the plan summary. The status bar for each test shows the recent build history of the test.

On this page:

To quarantine a failing test

You need plan administrator permission to quarantine a test.

  1. Choose Dashboard > All Plans > #buildresult to go to the build result where the test is failing.
  2. Click Quarantine for the failing test (in the 'Build Result Summary' screen).

To restore a quarantined test to a build

You need plan administrator permission to restore a test.

  1. Choose Dashboard and click on a plan to go to the plan's summary.
  2. Click the Quarantined Tests tab.
  3. Click Unleash for the test to be restored.

Screenshot: The quarantined tests for a plan, showing the Status bar.


  • No labels

8 Comments

  1. Anonymous

    If you have one or more quarantined tests, the outcome of the task will be changed to success if there are no unquarantined tests failing.  This can be a problem because if there is a compilation failure, the task will still be reported as a success!  This is currently a bug in Bamboo.  

    Fortunately, there is a work around to this issue.  Instead of using a single task to perform your build and parse the test results, use the JUnit parser task to parse the test results.  This will ensure that compilation failures will still fail your build.

  2. I'm not seeing the "Unleash" button on the Quarantined Tests page. Anyone have any idea why that might be?

    1. If you just have the plan admin privileges, the button will be missing as a result of this bug BAM-12319 - Plan Administrators cannot see Unleash buttons on Quarantined Tests page Resolved . It is now available only to users with gloabl admin permissions. Watch the bug report for its fix

       

  3. Bamboo might not fail the build, but maven still does.  Can this feature integrate with maven?  If not, then it has limited usefulness.

  4. Can quarantine be configured in such a way that if a test case is put in quarantine, it will not be executed by any other bamboo plan? Is a way to send a test case to the quarantine even if did cause a build failure

  5. I think that permissions should be changed so it does not require an admin to quarantine tests. 

    1. Anonymous

      I strongly agree:  developers should be able to quarantine a test.  On my project we never use this feature, even though it's intended to solve a problem we have all the time.  That's because developers are responsible for analyzing and fixing broken tests, and we have access to comment out a failed test but not to quarantine it.

      I wonder if Atlassian has really thought this through:  "In test-driven development (TDD), a test will fail until the functionality is implemented - you want to quarantine all but the relevant tests" and yet "You need plan administrator permission to quarantine a test."  Is it assumed that someone doing test-driven development generally has administrator permission?

    2. Agreed. User can't even access sibling functions "Create JIRA issue" and "Link with JIRA issue" without admin permission, that's inconvenient to say the least.