7. Test Optimization for IDEA

This page explains how to set up Clover's Test Optimization feature in the IDEA development environment.

On this page:

Before You Begin

Before using Test Optimization with Clover-for-IDEA, be aware of the following.

• Test Optimization is available as a 'Run Optimized' command, similar to 'Run' or 'Debug'.
• Test Optimization supports JUnit launch configurations only.
• Ensure you have Clover enabled on the project; when there is no Clover instrumentation, there is no Test Optimization.

Launching Test Optimization

To establish Test Optimization in Clover-for-IDEA, carry out one of the following actions:

• Right-click on a folder or package containing test classes and select ' Run Optimized', OR

Screenshot: Launching a Build with Test Optimization from the Context Menu

• Select an existing launch configuration in the Run Configurations drop-down menu and press icon.

Screenshot: Launching a Build with Test Optimization from the Drop-Down Menu

skitch

Measuring Test Optimization Results

When Optimized tests are being run, Clover displays additional info about savings (as shown in the screenshot below).

Screenshot: Pop-up Notification of Time Savings With Test Optimization

After Optimized tests run, Clover saves a snapshot file with coverage information that is used to optimize the following test runs.
This file may be deleted using the Delete Snapshot icon in the Clover Toolbar.

The Delete Snapshot icon is invisible when the project does not have the snapshot file.
Test Optimization would run all tests (no optimization) when the snapshot file is deleted or absent.

Test Optimization Settings

The list below shows the settings available for Test Optimization.

• Discard snapshot every X compiles: When enabled, snapshot is re-generated every X compiles. This is the equivalent of Ant's 'fullrunevery' setting (See Clover-for-Ant documentation).
• Minimize tests: main functionality. When disabled Clover only reorders tests, all of them are always run.
• Test reordering:
  • Do not reorder (means NOOP if Minimize Tests is also off)
  • Failing tests first: Re-order tests so that the ones which failed (the last time Optimized Test was run) are run first.
  • Random order.

Setting Global Preferences

Screenshot: Setting Global Preferences

These are global (IDE-scope) preferences of Test Optimization where it is defined how Clover notifies about empty test runs, i.e. ones that have all tests optimized out.

• Display a dialog: Shows a dialog box that requires the user to close it manually before test run can proceed,
• Display a balloon: Pops up a notification balloon.

Setting Per-launcher Preferences

Screenshot: Running Optimized Configurations

Test Optimization specific configuration options for specific JUnit configurations.
The defaults copied to new configurations may be set using Edit Defaults button.

Configurations Unsuitable For Test Optimization

Unfortunately not all configurations are suitable for Test Optimization. Please see the following points for specific details.

Limitations with Test Suites

Clover does not recognise test suites as entities that should be optimized away.
As the result test suites are always run (never optimized).
If your test launch configuration includes both test suite and the test case (which is probably an incorrect configuration), then the test case would be run twice (normal behavior) or once (via test suite) when the test case is optimized away.

Resolution: Do not include test suites in launch configuration, add test cases directly.

For more information, see this JIRA issue: CIJ-249.

Limitation with test methods

Due to bug CLOV-1084, in case when you execute optimized run and next add new test method to a class, such class will be always executed in next optimized runs. In order to fix it, you have to delete optimization snapshot (see button in tool bar menu).

Troubleshooting

To troubleshoot Test Optimization in Clover-for-IDEA, check through the following solutions:

1. If Clover is disabled for the project or generally Clover does not work for the project;
   
   • Ensure that the Clover icons are visible.
   • Check whether Coverage Explorer shows any coverage for the project.
2. If Clover has the test source settings wrong;
   
   • Check whether the Test Runs tool window shows any tests.
   • Ensure that Clover | Compilation | Instrument test source folders option is enabled.
3. If your test case is run twice, or not optimized at all;
   
   • Clover does not support test suites. Make sure you don't try to run one, launch test cases directly (CIJ-249).

Next chapter: 9. IDEA Advanced topics.