Running JUnit4 Parameterized tests with Surefire
Introduction
JUnit4 framework version 4.10 has introduced a feature, which allows to run the same test multiple times, using different data as input.
In order to use this, you have to:
- annotate test class with @RunWith(Parameterized.class)
- declare a data() method returning collection of input values and annotate this method with @Parameters annotation.
- declare a test method annotated with @Test
Furthermore, the JUnit version 4.11 has added a 'name' attribute to the @Parameters annotation - thanks to this, you can define a custom name for a test. You can use variables such as "{index}" for an iteration number and "{0}, {1}, ... " for N-th input argument in a test name.
Integrating Clover
Clover 3.3.0 introduced a JUnitTestRunnerInterceptor, which can be attached to JUnit's runner. It "listens" which test is being executed and what runtime name it has (evaluated by JUnit). Thanks to this, you can see an iteration number:
as well as full test names (@Parameters(name=...)) in the reports:
To enable this Clover extension, add the following snippet for maven-surefire-plugin:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<properties>
<!-- Attach Clover's test interceptor in order to record JUnit4 Parameterized tests -->
<property>
<name>listener</name>
<value>com.atlassian.clover.recorder.junit.JUnitTestRunnerInterceptor</value>
</property>
</properties>
</configuration>
</plugin>
</plugins>
LIMITATION
Clover's JUnitTestRunnerInterceptor can correctly handle parameterized test names when test methods from a single test case class are executed sequentially. It means that you shall not use a test runner which will run all iterations in parallel.
On the other hand, running entire test cases or test suites in parallel is allowed.
References
- Integrating Clover with JUnit4 Parameterized Tests - how to enable it programatically (useful for Ant and command line integration)
- Parameterized-tests - developer manual
- maven-surefire-plugin/examples/junit.html - about test listeners in Surefire
- https://bitbucket.org/atlassian/clover-examples - sample project with parameterized tests