Part 1 - Measuring Coverage
Welcome to the Clover-for-Ant tutorial. This document will walk you through the process of integrating Clover with an Ant build, gradually exploring Clover's more advanced code coverage features along the way.
On this page:
Introduction
Part one of this tutorial focuses on the creation and interpretation of Clover 'current' reports. Current reports display graphical and numerical data relating to the most recent coverage data collected for the project. This tutorial covers the initial creation of coverage data before stepping you through how to generate and interpret coverage reports. We'll then look at how to improve the coverage achieved by tests and regenerate the coverage reports. This section covers the very basic features of Clover and is an important first step for all users.
In this tutorial we will compile and unit-test the Money library provided in the tutorial/src
directory, then use Clover to determine how well the unit tests actually test the library.
In the first step, we will compile the Money library and run tests against it.
Step 1. Compiling and running
In this step we will compile the library and run the tests against it without using Clover to check that everything is working correctly before including Clover in the next step. In the tutorial
directory you will find the initial build file which contains targets for compiling, running and cleaning the build.
Compiling
To compile the java files, use the command ant compile
Output should be similar to the following:
$ ant compile
Buildfile: .../tutorial/build.xml
init:
[mkdir] Created dir: .../tutorial/lib
[get] Getting: https://repo1.maven.org/maven2/junit/junit/4.12/junit-4.12.jar
[get] To: .../tutorial/lib/junit-4.12.jar
compile:
[mkdir] Created dir: .../tutorial/build/classes
[javac] Compiling 3 source files to .../tutorial/build/classes
[mkdir] Created dir: .../tutorial/build/testclasses
[javac] Compiling 2 source files to .../tutorial/build/testclasses
BUILD SUCCESSFUL
This shows that the java source files have been compiled and class files have been placed in the tutorial\build
directory.
Running the tests
To run the JUnit tests, use the command ant test
Output should be similar to the following:
$ ant test
Buildfile: .../tutorial/build.xml
init:
[get] Destination already exists (skipping): .../tutorial/lib/junit-4.12.jar
compile:
test:
[mkdir] Created dir: .../tutorial/build/testresults
[junit] Running com.atlassian.samples.money.MoneyBagTest
[junit] Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.039 sec
[junit] Running com.atlassian.samples.money.MoneyTest
[junit] Tests taking too long? Try Clover's test optimization.
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.035 sec
BUILD SUCCESSFUL
This shows that all the tests have been run and have passed. We have now compiled the Money library, and ran tests against it. In the next step, we will add Clover targets and properties to the build file to enable measurement of code coverage.
Step 2. Adding Clover targets
Now that we have compiled the code and run unit tests, we are ready to add Clover targets and properties to the build file so we can measure the code coverage of the tests. Modifying the build file is trivial. Firstly we need to add a target to enable and configure Clover for the build.
Adding Clover task definitions
For this tutorial, ensure that the property clover.jar
has been defined as the path to your 'clover.jar' file. Hence, if you followed the Adding to Ant's build.xml instructions and have only added the Clover 'taskdef' resource to your 'build.xml' file, you'll need to redefine this resource to match the format described in this step.
Load the build.xml
file into your favorite text editor and add the Clover Ant task and type definitions:
<property name="clover.jar" location="../lib/clover.jar"/>
<taskdef resource="cloverlib.xml" classpath="${clover.jar}"/>
Note
This assumes that the clover.jar
is left in the unpacked Clover distribution from which this tutorial is being done. If you have installed the clover.jar
elsewhere, adjust the path accordingly.
These lines define the Clover Ant tasks which can then be used within the build file.
Adding a target to enable Clover
Add a target called with.clover
which will enable and configure Clover for a build:
<target name="with.clover">
<clover-setup/>
</target>
Adding Clover to the classpath
The clover.jar
needs to be in both compilation and runtime classpath. To achieve this, add the line in marked below to the build.classpath
Ant path:
<path id="build.classpath">
<pathelement path="${clover.jar}"/> <!-- add this -->
<pathelement path="${build.classes}"/>
</path>
Adding <clover-clean> to the clean target
It is advisable to add the <
clover-clean
/>
task to the clean
target. This will delete the Clover database when the clean
target is executed.
<target name="clean" >
<clover-clean/> <!-- add this -->
<delete dir="build"/>
</target>
Once you have made these changes, save the build.xml
file. We will add some more Clover targets later to perform coverage reporting, but first we will re-compile the Money library with Clover and re-run the tests to obtain coverage data.
Step 3. Testing with Clover
We are now ready to measure the coverage of the tests over the Money library.
Compile with Clover
Ensure that your build has been cleaned by running ant clean
. This deletes all class files from previous compilations.
Compile your code with Clover using the command ant with.clover compile
.
You will get output similar to the following:
$ ant with.clover compile
Buildfile: .../tutorial/build.xml
with.clover:
[clover-setup] Clover Version 4.1.0, built on ...
...
[clover-setup] Clover is enabled with initstring .../tutorial/.clover/clover4_1_0.db'
init:
...
compile:
[mkdir] Created dir: .../tutorial/build/classes
[javac] Compiling 3 source files to .../tutorial/build/classes
[clover] Clover Version 4.1.0, built on ...
[clover] Loaded from: .../lib/clover.jar
[clover] Clover: Site License registered to ...
[clover] Creating new database at '.../tutorial/.clover/clover4_1_0.db'.
[clover] Processing files at 1.8 source level.
[clover] Clover all over. Instrumented 3 files (1 package).
...
BUILD SUCCESSFUL
The result of this process is that your source files have been instrumented by Clover and then compiled as usual. As part of the instrumentation process, Clover creates a database that will be used during the coverage recording and report process.
Running the tests
We now need to run the tests again, using the command ant test
. This command will run the tests, this time measuring coverage. Output from Ant will be the same as a normal test run:
$ ant test
Buildfile: .../tutorial/build.xml
init:
...
compile:
test:
[mkdir] Created dir: .../tutorial/build/testresults
[junit] Running com.atlassian.samples.money.MoneyBagTest
[junit] Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec
[junit] Running com.atlassian.samples.money.MoneyTest
[junit] Tests taking too long? Try Clover's test optimization.
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.063 sec
BUILD SUCCESSFUL
During this test run, Clover measured the code coverage of the tests and wrote the coverage data to disk.
In the next step we'll generate a coverage report from this data to see how well the tests actually cover the Money library.
Step 4. Creating a report
We are now ready to produce a coverage report. This section will focus on producing a Clover HTML report. For information on how to generate a PDF report see the <
clover-pdf-report
>
task, or for other types of Clover reports see the <
clover-report
>
task.
Adding a Clover report target
Open the build.xml
file in a text editor and add the following target
to create a HTML report:
<target name="clover.report">
<clover-html-report outdir="build/clover_html" title="Clover Demo"/>
</target>
The <
clover-html-report
>
task is a simplified version of the <
clover-report
>
task. As no historydir
attribute has been specified, it uses the current coverage data. Historical reports, which show the progress of coverage over the life of the project, are discussed later in this tutorial (see Part 2 - Historical Reporting). The current report is to be in HTML format, written to the directory build/clover_html
and with the title "Clover Demo
". The output directory build/clover_html
is relative to the path of the Ant build file. In this case, the directory build/clover_html
will be nested within tutorial
as this is the location of build.xml
.
Generating the report
Create a HTML report with the command ant clover.report
. You will get output similar to the following:
$ ant clover.report
Buildfile: /Users/mparfianowicz/Work/clover-hg/tutorial/build.xml
clover.report:
[clover-html-report] Clover Version 4.1.0, built on ...
[clover-html-report] Loaded from: .../lib/clover.jar
[clover-html-report] Clover: Site License registered to ...
[clover-html-report] Loading coverage database from: '.../tutorial/.clover/clover4_1_0.db'
[clover-html-report] Writing HTML report to '.../tutorial/build/clover_html'
[clover-html-report] Done. Processed 1 packages in 1253ms (1253ms per package).
BUILD SUCCESSFUL
You can now view the report by opening the file tutorial/build/clover_html/index.html
in a web browser. See 'Current' Report for details about interpreting this coverage report.
In the next step, we will enhance the JUnit tests to improve code coverage of the Money library.
Step 5. Improving coverage
After having a look at the coverage report, you'll notice that coverage is not 100%. Although not always possible, it is best to get as close to full coverage as you can. Think of it this way: every line that isn't covered could contain a bug that will otherwise make it into production. You should certainly aim to cover all of the code that will be executed under normal operation of the software.
One method in the Money library that is not fully covered is the equals()
method in the Money class (lines 40-42 as seen below). The first few lines of this method handle the special case when the Money value is zero. The coverage report shows that the code to handle this has not been covered by the tests. Line 40 has been executed 27 times, but since it has never evaluated to true
it has not been fully covered and is therefore in red. It follows then that the two successive lines have never been executed.
We can now improve the tests so that this section of code is covered. To do this, make the following additions (shown in bold) to the MoneyBagTest.java
file.
Declare the variable f0USD
:
public class MoneyBagTest {
private Money f12CHF;
private Money f14CHF;
private Money f7USD;
private Money f21USD;
private Money f0USD;
...
Initialize f0USD
in the setUp()
method:
public void setUp() {
f12CHF = new Money(12, "CHF");
f14CHF = new Money(14, "CHF");
f7USD = new Money( 7, "USD");
f21USD = new Money(21, "USD");
f0USD = new Money(0, "USD");
...
Finally, the following test needs to be added:
public void testMoneyEqualsZero() {
assertTrue(!f0USD.equals(null));
IMoney equalMoney = new Money(0, "CHF");
assertTrue(f0USD.equals(equalMoney));
}
After these amendments have been made, compile and run tests again (by running ant test
), then re-generate the HTML report (by running ant clover.report
). You will see that the Money class now has 100% coverage.