Test reporting in Pipelines

On this page

Still need help?

The Atlassian Community is here for you.

Ask the community

We've made it easier to quickly identify failed tests in your build. Test reporting will be automatically enabled in your pipeline when it detects xUnit-compatible test results which include a fail.

How test reporting works

If your build step generates test results, pipelines will automatically detect and show any failures in the web interface. Pipelines work with any XML xUnit-compatible test reports, regardless of the language they are written in. 

Any user with read access to the repository can view the test results in the pipelines log panel. If failed test results are found, the log view will change to highlight the failed tests, showing the reason and the stack-trace if present. The full log view is also available using the Build tab.

Note: the Tests tab will only show if you have failed tests to review.

Screenshot of test log view

Configuring test reporting

To enable test reporting, make sure that build test reports are generated in one of the supported default locations (with a directory depth of 3 levels):

./**/surefire-reports/**/*.xml
./**/failsafe-reports/**/*.xml
./**/test-results/**/*.xml
./**/test-reports/**/*.xml


The test report file scanner will begin searching from the base directory of your build: /opt/atlassian/pipelines/agent/build

Configuring test runners

Maven Surefire / Failsafe plugins

For Maven build jobs, no special configuration is required if you are using Maven Surefire Plugin or Maven Failsafe Plugin. The reports are automatically generated when the unit or integration maven test goals are executed.

Gradle

For Gradle builds, test reports are automatically generated in XML xUnit-compatible format. No special configuration is required as, by default, Gradle generates the test results in one of the default locations.

PHPUnit

For PHPUnit test reports, you should explicitly specify the --log-junit parameter to generate the test reports output to a particular location.

bitbucket-pipelines.yml
image: php:7.1.1
pipelines:
  default:
    - step:
        script:
          - apt-get update && apt-get install -y unzip
          - curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
          - composer require phpunit/phpunit
          - vendor/bin/phpunit --log-junit ./test-reports/junit.xml

Mocha for Node.js

For Mocha test framework, you can use the mocha-junit-reporter and configure the output file where the test report will be generated. 

bitbucket-pipelines.yml
image: node:6
pipelines:
  default:
    - step:
        script:
          - npm install mocha mocha-junit-reporter --save-dev
          - mocha test --reporter mocha-junit-reporter --reporter-options mochaFile=./test-reports/junit.xml

Unittest for Python

For Python Unittest framework you can use the unittest-xml-reporting module when configuring the tests, adding the XMLTestRunner in your test configuration.

test.py
import unittest
import xmlrunner

class TestExample(unittest.TestCase):
    def testIsTrue(self):
        self.assertTrue(True)

if __name__ == '__main__':
    unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))

Other test runners

For other test runners make sure that the XML xUnit-compatible test results are generated as a part of your build in one of the default supported locations.



Last modified on Nov 20, 2018

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.