This page explains Clover's Test Optimization feature and gives a brief explanation of how it works. See the Test Optimization Quick Start Guide for practical instructions.
On this page:
|Table of Contents|
What is Test Optimization?
Test optimization will make a build potentially complete a lot faster than a full build and test run. It should do this without substantially compromising the quality of the feedback it gives; in other words a quicker pass or fail result, but a reasonably accurate pass or fail.
There are two ways of ensuring a build completes quickly:
1. Run only the tests required to confirm the validity of the changes that triggered the build.
2. Run all the tests but in an optimal order: any failed tests from the previous build, all tests covering modified code, then in ascending order by test invocation time.
Since Clover records which tests covered which lines of code, it can tell the build to only run tests that cover code modified since the last build.
How Test Optimization Works
The following is a general outline of what is required to enable Clover to optimize the test and build process.
- A full clean build is performed. Any existing Clover databases are removed.
- Clover performs instrumentation on all Java source files. The Clover registry is created.
- javac compiles the instrumented sources
- All unit tests are run. Coverage data is stored next to the Clover registry.
- A Clover report may optionally be produced.
- A Clover snapshot is saved - this is essentially a mapping of application source files to the set of tests which hit each file.
- Zero or more Java source files are modified, added or removed
- Clover re-instruments either only the modified source files or all source files (depending on how Clover is invoked). The Clover registry is updated, and any files modified or added since the last snapshot are marked as such.
- Clover uses this information and the snapshot from the previous test run to determine which tests need rerunning.
- The test runner is invoked. Only tests obtained in #9 and any tests unknown to Clover (perhaps excluded from instrumentation) are run. Tests are ordered to encourage early failures.
- A snapshot is saved.
- Go to step 7 unless a maximum number of optimised builds has reached,
clover.jarhas changed between builds or some other build-specific condition signifies a full rebuild is required (e.g. configuration file changes) in which case go to step 1.