Part 2 - Historical Reporting
Part two of this tutorial focuses on the creation and interpretation of Clover 'Historical' reports.
On this page:
Historical reports display graphical and numerical data relating to sets of coverage data collected over time for the project. This tutorial covers the generation of a set of historical data, interpretation of the information displayed in the Historical reports and customization of the reports for your particular reporting preferences.
In the first step, we will edit the Ant build file to generate a history point.
A history point is a snapshot of code coverage and metrics data for the project at a particular point in time. By running tests with Clover over time and creating a series of history points, it is possible to compare code coverage and metrics by viewing results in a single Clover report and enabling you to track the development of your project.
Step 1 - Creating history points
Step 1 describes how to set up the relevant Ant target and run the command so that a history point can be created. The generation of historical reports is discussed in later steps.
Adding a history point target
Add the following target to your build.xml file:
<target name="record.point">
<clover-historypoint historyDir="clover_history"/>
</target>
When this target is run, a history point will be created with the timestamp value of the coverage run.
The value of the historyDir parameter is the directory where the history points will be stored. You should create this directory before executing this target.
By default, Clover records the history point with a timestamp of the coverage run. If you wish to override the timestamp value of a history point, you can add date and dateformat attributes to the task allowing you to reconstruct coverage history. See documentation for the <clover-historypoint> task for details.
Recording a history point
Ensure that the source code has been instrumented, and the tests run, by using the command ant clean with.clover test.
Run the command ant record.point. Output should be similar to the following:
$ ant record.point
Buildfile: .../tutorial/build.xml
record.point:
[clover-historypoint] Clover Version 4.1.0, built on ...
[clover-historypoint] Loaded from: .../lib/clover.jar
[clover-historypoint] Clover: Site License registered to ...
[clover-historypoint] Writing report to '.../tutorial/clover_history/clover-20151029123507.xml.gz'
BUILD SUCCESSFULIn the next step we will add more tests to improve coverage of the Money library, recording Clover history points along the way.
Step 2 - Generating historical data
In Part 1 of the tutorial we made additions to the testing suite to improve code coverage. In order to show the historical reporter in use, we will now continue to add tests and periodically record history points which will later be used as code coverage and metrics data by the historical reporter.
The Money.java file is at 100% coverage, but there are several sections of code that remain untested in the MoneyBag.java file. This step will focus on bringing the coverage of this class to 100%, as well as creating historical data in the form of history points.
Open the source file MoneyBagTest.java in your favorite text editor and make the following additions shown in bold:
Declare the variables f0CHF and fMB3:
public class MoneyBagTest extends TestCase {
private Money f12CHF;
private Money f14CHF;
private Money f7USD;
private Money f21USD;
private Money f0USD;
private Money f0CHF;
private IMoney fMB1;
private IMoney fMB2;
private IMoney fMB3;
...
Initialize f0CHF and fMB3 in the setUp() method:
protected void setUp() {
f12CHF = new Money(12, "CHF");
f14CHF = new Money(14, "CHF");
f7USD = new Money( 7, "USD");
f21USD = new Money(21, "USD");
f0USD = new Money(0, "USD");
f0CHF = new Money(0, "CHF");
fMB1 = MoneyBag.create(f12CHF, f7USD);
fMB2 = MoneyBag.create(f14CHF, f21USD);
fMB3 = MoneyBag.create(f0CHF, f0USD);
...
Add the following test:
public void testMoneyBagEqualsZero() {
assertTrue(!fMB3.equals(null));
IMoney expected = MoneyBag.create(new Money(0, "CHF"), new Money(0, "USD"));
assertTrue(fMB3.equals(expected));
}
After making the above changes, re-instrument and test your code by running ant clean with.clover test. Then record a new history point by running ant record.point. By recording new history point now, Clover will capture the new state of code coverage and metrics for comparison with past or future runs.
Add the following tests to bring the coverage of the Money project to 100%:
public void testToString() {
String expected="{[12 CHF][7 USD]}";
assertEquals(expected, fMB1.toString());
}
public void testVectorSize() {
IMoney other = MoneyBag.create(new Money(2, "CHF"), new Money(2, "USD"));
assertTrue(!other.equals(fMB3));
}
Once again, re-instrument your code, test and record a new history point.
We have now created a series of history points for the Money library. The next section discusses how to generate a Clover historical report which will display the historical data that has been collected.
Step 3 - Creating historical reports
Now that we have recorded several history points, the next step is to add a target to the build file which will call the historical reporter and generate a historical report.
Add a historical report target
Add the following target to build.xml:
<target name="hist.report">
<clover-report>
<historical outfile="historical.pdf"
historyDir="clover_history">
<format type="pdf"/>
</historical>
</clover-report>
</target>
The hist.report target is similar to the report.html target defined in Part 1. The main differences are that the nested element specifies <historical> rather than <current> and there is no specification of the output format as html.
The historical reporter needs to be able to find the coverage history files in order to create the report; so the historyDir value must be the same as the historyDir defined for the history points. The format of the report can be either PDF or HTML, as specified by the <format> element <clover-report>. The <format> element is optional and is not included in the example above. When the <format> element is omitted, a PDF report is produced by default. Depending on the chosen format, the outfile value may represent a single file as in the case of the PDF format, or the name of a directory (in the case of the HTML format).
Generating a historical report
Create a historical report by using the command ant hist.report. Output should be similar to the following:
$ ant hist.report
Buildfile: /Users/mparfianowicz/Work/clover-hg/tutorial/build.xml
hist.report:
[clover-report] Clover Version 4.1.0, built on ...
[clover-report] Loaded from: .../lib/clover.jar
[clover-report] Clover: Site License registered to ...
[clover-report] Loading historical coverage data from: '.../tutorial/clover_history'
[clover-report] Read 3 history points.
[clover-report] using movers interval of 57 seconds
[clover-report] using movers interval of 57 seconds
BUILD SUCCESSFUL The report can now be viewed by opening the file tutorial/historical.pdf in a PDF viewer such as Adobe Acrobat Reader.
To interpret this history report, see Understanding a Historical Report
Step 4 - Customizing historical reports
In the previous sections of this tutorial we have looked at how to create and interpret a basic historical report. In addition to basic reporting, the historical reporter is highly configurable and this section will detail some of the options you can use. For a full list of the report configuration options, see the documentation for the <clover-report> task.
Changing output format
The default historical report type is PDF, although an HTML report can also be produced. To create an HTML report, add a nested <format> element with type specified as html to your <clover-report> element. Try adding the following target to your build.xml file and executing the command ant hist.report.html:
<target name="hist.report.html">
<clover-report>
<historical outfile="build/clover_html/historical"
title="My Project"
historyDir="clover_history">
<format type="html"/>
</historical>
</clover-report>
</target>
A custom title can also be displayed for your report by using the title attribute in the <historical> element as above. You can find report in tutorial/build/clover_html/historical/historical.html.
Chart selection
The historical reporter allows you to specify which charts to include in your report, and also allows you to configure further options in the charts themselves.
The default reporting mode is to include all four report elements: <overview>, <coverage>, <metrics> and <movers>. But to include some and not the others is a simple matter of nesting the desired elements within the <historical> element. Try adding the following target to your build.xml file as an example:
<target name="hist.report.coverage">
<clover-report>
<historical outfile="histCoverage.pdf"
title="My Project"
historyDir="clover_history">
<overview/>
<coverage/>
<format type="pdf"/>
</historical>
</clover-report>
</target>
The above code will produce a historical PDF report with the title 'My Project' which includes only two sections: the 'Overview' and the 'Coverage over time' charts.
Chart configuration
Clover presents flexible charting configuration, allowing you to present information exactly as you like it. The <chart> element allows you to define a custom chart and fill it with specific data with the <columns> element.
<target name="hist.report.select">
<clover-report>
<historical outfile="histSelect.pdf"
title="My Project"
historyDir="clover_history">
<chart/>
<metrics/>
<format type="pdf"/>
</historical>
</clover-report>
</target>
This will produce a PDF file with the filename 'histSelect.pdf' with two sections: the a custom chart with total coverage information; and the 'Metrics over time' chart. You can also specify whether or not a chart uses a log scale by adding the logscale attribute:
<metrics logscale="false"/>
'Movers' configuration
The 'Movers' section of the historical report shows you the classes whose coverage has changed the most recently. This is useful for spotting classes that have had sudden changes in coverage, perhaps the unintended result of changes to the unit test suite.
The 'Movers' chart allows you to specify the threshold of point change a class must satisfy, the maximum number of gainers and losers to display and the period across which the gains and losses are calculated. Add the following target to your build.xml file as an example of this feature in use:
<target name="hist.report.movers">
<clover-report>
<historical outfile="histMovers.pdf"
title="My Project"
historyDir="clover_history">
<movers threshold="5%" range="20" interval="2w"/>
<format type="pdf"/>
</historical>
</clover-report>
</target>
In this case, the configuration values selected state that classes must have a change in coverage of at least 5 percentage points to be included in the chart, a maximum of 20 gainers and 20 losers can be displayed, and the initial valuation point for class coverage is 2 weeks prior to the most recent history point. Should there be greater than 20 gainers in this period, then the classes with the biggest percentage point gain will be displayed, and the same for the losers.
See Interval Format for details on the syntax for specifying interval values.
The next section of this tutorial will discuss how you can automate the coverage checking of your project.