Introduction

This is a tutorial showing how to use Clover:

• with projects containing multiple modules,
• running on multiple application severs,
• in multiple test phases (e.g. unit tests, integration, manual testing),
• in multiple test runs (snapshots and history points)
• in distributed environment

This tutorial shall help you answering questions like:

• should I use cloverDatabase, singleCloverDatabase or cloverMergeDatabase?
• should I deploy instrumented code on all application servers and run tests at once or sequentially?
• what should I copy (or not copy) to test server?
• should I use distributed.coverage=on?
• should I merge Clover databases?

Decision matrix

Step 1: Building your application with Clover

Step 2: Running / testing your application with Clover

Step 3: Aggregating coverage data and generating report

Optional: Optimizing your environment configuration

Optional: Per-test coverage and test optimization and distributed coverage

Assumptions for all scenarios:

For the simplicity of the tutorial it's assumed that:

• we have separated machines for build, tests and reporting
• we have a shared network drive accessible from all machines at the same absolute path

In case when your environment is different and:

• build, application or reporting server is the same machine => skip points talking about copying files
• shared network drive is accessible from all machines but at different absolute paths => instrument sources using relative paths, use clover.initstring.basedir / clover.initstring.prefix at runtime
• shared network drive is not available => copy clover.db / recording snapshots / history points between machines

Example #1 - one application executed on several servers

Assumptions

• there is one application having one Clover database
  (e.g. a Maven project with a <singleCloverDatabase> or an Ant project with one <clover-setup>)
• application is deployed to several machines
• we're not interested in per-test coverage recording
• we want to have a single report showing combined coverage from all machines

TIP: this scenario applies also to a case when:

• application is deployed on one server or
• application is deployed on one server and running in multiple JVMs or
• application is executed multiple times (for example: unit tests + integration tests + manual testing)

Overview chart

Steps

1) Build application with Clover

a) using Ant

Define initstring attribute for <clover-setup> or <clover-instr> tag, e.g.:

<clover-setup initstring="/path/to/network/drive/clover.db" />

b) using Maven

Define <cloverDatabase> and <singleCloverDatabase> property in pom.xml, e.g.:

<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <singleCloverDatabase>true</singleCloverDatabase> 
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
    </configuration>
</plugin>

mvn clean clover:setup install

2) Deploy instrumented application

Copy clover.jar and your application EAR/WAR. There's no need to copy clover.db as it's on a network drive.

3) Run tests

Execute your tests. There is no need to provide clover.initstring parameter at runtime as path to clover.db database is already hardcoded in instrumented sources.

4) Generate coverage report

a) using Ant

Execute <clover-report> task; use initstring pointing to clover.db on a network drive, e.g.:

<clover-report initstring="/path/to/network/drive/clover.db">

b) using Maven

Execute clover:clover goal. Note that you don't need to call clover:aggregate or clover:merge as there is only one database. The <cloverDatabase> defined in pom.xml will be used for reporting.

<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <singleCloverDatabase>true</singleCloverDatabase>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
    </configuration>
</plugin>

mvn clover:clover

Example #2 - one application with many Clover databases executed on several servers

Assumptions

• one application containing multiple Clover databases
  
  • like a Maven project with many modules, each with own clover.db,
  • or an Ant project with several modules, each with own clover.db
• an application is deployed to several machines
• we're not interested in per-test coverage recording
• we want to have a single report showing combined coverage for all modules from all servers

Overview chart

Steps

1) Build application with Clover

a) using Ant

<target name="all">
   <!-- Enable Clover for every module, each with own database -->
   <clover-setup initstring="/path/to/network/drive/moduleX/clover.db">
</target>

b) using Maven

• don't define singleCloverDatabase and cloverDatabase, so that default values will be used
• keep databases on a shared drive (or copy them to test machines)

<!-- top level pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <!-- no <cloverDatabase> and <singleCloverDatabase> -->
    </configuration>
</plugin>

2) Deploy instrumented application

Copy clover.jar and your application EAR/WAR. There's no need to copy clover.db files as they're on a network drive.

3) Run tests

Execute your tests. Provide clover.initstring.basedir=/path/to/top-level-module/dir or use clover.initstring.prefix at runtime. There is no need to define such property if databases are available under the same absolute path on test machines as on the build machine.

4) Generate coverage report

Before generating report you have to merge all databases. For example:

a) using Ant

<clover-merge initstring="/path/to/mergedClover.db">
   <cloverDb initstring="/path/to/network/drive/moduleA/clover.db"/>
   <cloverDb initstring="/path/to/network/drive/moduleB/clover.db"/>
</clover-merge>
<clover-report initstring="/path/to/mergedClover.db"/> 

b) using Maven

<!-- Top-level pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <cloverMergeDatabase>/path/to/network/drive/cloverMerged.db</cloverMergeDatabase>
    </configuration>
</plugin>

mvn clover:aggregate clover:clover

Example #3 - multiple applications executed on several servers in isolation

Assumptions

• we have multiple applications (each of them can have one or more modules)
  
  • every application has separate code base (i.e. no shared source files)
  • every application is built and instrumented separately (i.e. separate clover.db for every app)
• we deploy instrumented applications to several application servers
  
  • every application runs on it's own server (i.e. no case when two apps runs in the same JVM)
• tests run independently on each application server and for each application
• we're not interested in per-test coverage recording
• we want to have a single report showing combined coverage for all modules of all applications from all application servers

Overview chart

Steps

1) Build application with Clover

a) using Ant

Define initstring attribute for <clover-setup> or <clover-instr> tag, which will point to different directory for every application ,e.g.:

<!-- App1 build.xml -->
<clover-setup initstring="/path/to/network/drive/app1/clover.db">

<!-- App2 build.xml -->
<clover-setup initstring="/path/to/network/drive/app2/clover.db">

b) using Maven

Define <cloverDatabase> property for Clover plugin in pom.xml. You can use singleCloverDatabase in case your application is multi-module. Databases for all applications must be stored under common root (it's a limitation of clover:merge goal; Ant clover-merge task is more flexible regarding paths). Example:

<!-- App1 pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/common-root/app1/clover.db</cloverDatabase>
    </configuration>
</plugin>

<!-- App2 pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/common-root/app2/clover.db</cloverDatabase>
        <singleCloverDatabase>true</singleCloverDatabase> <!-- assuming that app2 is multi-module -->
    </configuration>
</plugin>

mvn clean clover:setup install

2) Deploy instrumented application to Application Servers

Copy clover.jar and your application jar/war to proper machines. There's no need to copy clover.db as it's on a network drive.

3) Run tests on Application Servers

Execute your applications. As every application runs in their own JVM and due to fact that we have used cloverDatabase (and optionally singleCloverDatabase) pointing to absolute path on a network drive, we don't need to provide clover.initstring parameter at runtime, because correct path is hardcoded in instrumented classes.

4) Generate coverage report

Before generating report you have to merge all databases. for example:

a) using Ant

<clover-merge initstring="/path/to/network/drive/cloverMerged.db">
   <cloverDb initstring="/path/to/network/drive/app1/clover.db"/>
   <cloverDb initstring="/path/to/network/drive/app2/clover.db"/>
</clover-merge>
<clover-report initstring="/path/to/cloverMerged.db"/> 

b) using Maven

<!-- Top-level pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <cloverMergeDatabase>/path/to/network/drive/cloverMerged.db</cloverMergeDatabase>  <!-- output database -->
        <baseDir>/path/to/network/drive/common-root</baseDir>  <!-- common root -->
        <includes>*.db</includes> <!-- filename pattern, separated by comma or space -->
    </configuration>
</plugin>

mvn clover:aggregate             # run it for multi-module applications
mvn clover:merge clover:clover  # run it for final report

Example #3b - multiple applications executed on several servers in partial isolation

This scenario is a variation of Scenario 3 in such way, that:

• location of clover.db(s) on test server is different than on build server (so you have to provide/change initstring at runtime)
• some applications are executed in the same JVM (as a consequence, you cannot pass clover.initstring as JVM argument, because you need a different value for each application).

In such case, you have to:

• instrument these applications using relative paths in <cloverDatabase> parameter (Maven) or <clover-setup initstring=""> (Ant), like below:

app1/clover.db
app2/clover.db
app2/moduleA/clover.db
app2/moduleB/clover.db

• copy generated clover.db(s) to test server, keeping their relative paths (under a common root), for instance:

/path/to/common-root/app1/clover.db
/path/to/common-root/app2/clover.db
/path/to/common-root/app2/moduleA/clover.db
/path/to/common-root/app2/moduleB/clover.db

• provide clover.iniststring.basedir=/path/to/common-root at runtime

Example #4 - multi-module application with distributed execution on several servers

Assumptions

• we have a multi-module project (like Maven project with modules, or Ant project with several build.xml files with inheriting properties across ant calls)
• we deploy instrumented application (the same code) to several application servers
• unit tests are executed on one <<server>> machine, but these unit tests call business logic on one or more <<client>> machines
• we are interested in per-test coverage recording, showing combined coverage from distributed execution
• we want to have a single report showing combined coverage for all modules from all application servers

Overview chart

Steps

1) Build application with Clover

• define how many clients will connect to JVM running unit tests;

a number greater than 0 means that server will hold until all clients are connected before it continues execution; number equal 0 means that tests will start immediately

you might have a dependency loop so that server waits for clients and clients wait for server - see below

• define server host name (default is localhost) and listening port (default is 1198)
• optionally define connection timeout (in milliseconds), retry period 

a) using Ant

<clover-setup initstring="/path/to/network/drive/clover.db">
    <distributedCoverage host="my.server.com" port="1234" numClients="2" timeout="10000"/>
</clover-setup>

b) using Maven

<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
        <singleCloverDatabase>true</singleCloverDatabase>  <!-- In case of multi-module application (optional) -->
        <distributedCoverage>
            <host>my.server.com</host>
            <port>1234</port>
            <numClients>2</numClients>
            <timeout>10000</timeout>
        </distributedCoverage>
   </configuration>
</plugin>

2) Deploy instrumented application to Application Servers

Copy clover.jar and your application jar/war to proper machines. There's no need to copy clover.db as it's on a network drive.

3) Run tests on Application Servers

On <<server>> machine

java ... -Dclover.server=true

On <<client>> machines

You don't have to provide any runtime options for JVM. They're already compiled in the code.

Potential problems

4) Generate coverage report

a) using Ant

<clover-report initstring="/path/to/network/drive/clover.db">
    <current showUniqueCoverage="true" outfile="/path/to/clover/report">
        <format type="html"/>
        <fileset dir="src"/>
    </current>
</clover-report>

b) using Maven

In order to show per-test coverage in the HTML report (showUniqueCoverage), you have to use the custom <reportDescriptor> in pom.xml and in the report descriptor set the showUniqueCoverage=true. For example:

<project name="Clover Report" default="current">
    <clover-setup initString="${cloverdb}"/>
    <target name="historical"/>
    <target name="current">
        <clover-report>
            <current showUniqueCoverage="true" outfile="${output}">  <!-- Show per-test coverage in report -->
                <format type="html"/>
            </current>
        </clover-report>
    </target>
</project>

<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
        <outputDirectory>/path/to/clover/report</outputDirectory>
        <reportDescriptor>report-descriptor.xml</reportDescriptor>    <!-- Use custom report -->
    </configuration>
</plugin>

mvn clover:clover

Example #5 - multi-module application with history points

Assumptions

• we have a multi-module project (like Maven project with modules, or Ant project with several build.xml files with inheriting properites across ant calls)
• we deploy instrumented application to one application server
• we are not interested in per-test coverage or test optimization
• we want to have a single report showing combined coverage for all modules

Overview chart

Steps

 1) Build application with Clover

a) using Ant

<target name="all">
   <!-- Enable clover for top level module -->
   <clover-setup initstring="/path/to/network/drive/clover.db">
   <!-- Build sub-modules ensuring that properties are passed -->
   <ant inheritrefs="true" inheritprops="true" file="sub-module-a/build.xml" target="all"/>
   <ant inheritrefs="true" inheritprops="true" file="sub-module-b/build.xml" target="all"/>
</target>

b) using Maven

<!-- TOP LEVEL POM.XML -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
        <singleCloverDatabase>true</singleCloverDatabase>
    </configuration>
</plugin>

mvn clover:setup test 

2) Deploy instrumented application to Application Servers

Remove previous version of application and copy clover.jar and your application jar/war. There's no need to copy clover.db as it's on a network drive.

3) Run tests on Application Servers

Execute your application. As we have used singleCloverDatabase and cloverDatabase pointing to absolute path on a network drive, we don't need to provide clover.initstring parameter at runtime.

4) Generate coverage report

a) using Ant

<clover-report initstring="/path/to/network/drive/clover.db"> 
    <current outfile="/path/to/clover/report/current" title="Coverage Report">
        <format type="html"/>
        <fileset dir="src"/>
    </current>
    <historical outfile="/path/to/clover/report/historical" title="Historical Report" historyDir="/path/to/clover/historypoints">
       <format type="html"/>
    </historical>
</clover-report>

<clover-historypoint historyDir="/path/to/clover/historypoints">
       <fileset dir="src"/>
</clover-historypoint>

b) using Maven

<!-- Top-level pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>clover-maven-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
        <singleCloverDatabase>true</singleCloverDatabase>
        <generateHistorical>true</generateHistorical>
        <generateHtml>true</generateHtml>
        <historyDir>/path/to/clover/historypoints</historyDir>
        <outputDirectory>/path/to/clover/report</outputDirectory>
    </configuration>
</plugin>

mvn clover:clover clover:save-history

References

• About Distributed Per-Test Coverage
• Working with Distributed Applications
  
  • Using Distributed Per-test Coverage with Clover-for-Ant
  • Using Distributed Per-test Coverage
• Using Clover for web applications
• FAQ / Atlassian Answers
  
  • When using Clover, why do I get a java.lang.NoClassDefFoundError when I run my code?
  • No coverage being generated from WebLogic hosted instrumented EAR
  • Clover on J2EE: Manual Testing

Don't define <cloverDatabase> and <singleCloverDatabase> so that default values will be used (relative path target/clover/clover.db and false).