Updating optimization snapshot file

Introduction

Imagine a following scenario:

• your project has a set of integration tests which follow a classic maven-failsafe-plugin approach:
  
  • tests are being executed in the 'integration-test' phase
  • but results are checked in the 'verify' phase
• test cases are being recognized by Clover
  
  • ... just because they are JUnit / TestNG test cases or you have defined custom test patterns for clover2:setup MOJO
  • ... you can see them on the "Tests"
• you would like to use test optimization for them
  • i.e. re-run only those which were failed or related sources were modified
• but Clover does not see test failures for them (in the clover.snapshot) and thus you cannot optimize your tests correctly
  
  • this is because test case is executed in 'integration-test' phase but it does not throw any exception (like AssertionError for JUnit) because verification is performed in later phase

How to solve this?

Solution

You can update Clover's Optimization Snapshot file "manually". Exact integration would depend on the framework you are using and you need to write such integration, but in general it works this way:

1) Instrument code and run integration-tests (mvn clean clover:setup integration-test)

2) Save optimization snapshot file (clover2:snapshot)

3) Run test result verification (mvn verify) and store failed test results (somewhere)

4) Update optimization snapshot file and set status for failed tests (see code example below)

Example

A simple application which sets test duration and test failure for certain test case:

https://bitbucket.org/atlassian/maven-clover2-plugin (src/it/optmized/snapshot-hacking)