Introduction

This is a "multi-multi" tutorial showing how to use Clover:

• with projects containing multiple modules,
• running on multiple application severs,
• in multiple test phases (e.g. unit tests, integration, manual testing),
• in multiple test runs (snapshots and history points)
• in distributed environment

If you have questions like:

• should I use cloverDatabase, singleCloverDatabase or cloverMergeDatabase?
• should I declare Clover in master pom.xml or in child modules' pom.xmls as well?
• should I deploy instrumented code on all application servers and run tests at once or sequentially?
• what should I copy (or not copy) to test server?
• should I use distributed.coverage=ON?
• should I use merge clover databases?

then this tutorial is for you 

Assumptions for all scenarios:

For the simplicity of the tutorial it's assumed that:

• we have separated machines for build, tests and reporting
• we have a shared network drive accessible from all machines at the same absolute path

In case when your environment is different and:

• build, application or reporting server is the same machine => skip points talking about copying files
• shared network drive is accessible from all machines but at different absolute paths => instrument sources using relative paths, use clover.initstring.basedir / clover.initstring.prefix at runtime
• shared network drive is not available => copy clover.db / recording snapshots / history points between machines

Scenario 1 - single-module application executed on several servers independently

Assumptions:

• we have a single-module application (like maven project with one pom.xml or ant project with one build.xml)
• we deploy instrumented application (the same code) to several application servers
• tests run independently on each application server
• we're not interested in per-test coverage recording
• we want to have a single report showing combined coverage from all application servers

TIP: this scenario applies also to a case when

• you deploy the application on one server or
• you deploy the application on one server and run multiple instances in separate JVMs or
• you execute your application multiple times (for example unit tests + integration tests + manual testing)

Overview chart:

Solution approach:

1) because of fact that

• we're not interested in per-test coverage AND
• we run tests independently

there is no need to configure a distributed coverage feature.

2) because of fact that

• we deploy the same code to all application servers AND
• we want to have a single report showing combined coverage from all application servers

we have the same code base, so the same clover.db should be used on each application server and merging clover databases has no sense

we should keep coverage recordings from all servers in the same location as well - note that they will not overwrite each other, because every snapshot file has unique file name

3) because of fact that

• we have a shared network drive accessible from all machines at the same absolute path

it will be very convenient to use it to store clover.db as well as recordings

we will not need to use -Dclover.initstring at runtime, because the absolute path will be hardcoded in the instrumented code

Steps:

1) Build application with Clover

a) using Ant

Define initstring attribute for <clover-setup> or <clover-instr> tag, e.g.:

<clover-setup initstring="/path/to/network/drive/clover.db">

b) using Maven

Define <cloverDatabase> property for Clover plugin in pom.xml, e.g.:

<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
    </configuration>
</plugin>

mvn clean clover2:setup install

2) Deploy instrumented application to Application Servers

Copy clover.jar and your application jar/war. There's no need to copy clover.db as it's on a network drive.

3) Run tests on Application Servers

Execute your application. There is no need to provide clover.initstring parameter as path to clover.db database is already hardcoded in instrumented sources.

4) Generate coverage report

a) using Ant

Execute <clover-report> task; use initstring pointing to clover.db on a network drive, e.g.:

<clover-report initstring="/path/to/network/drive/clover.db">

b) using Maven

Execute clover2:clover goal. Note that you don't need to call clover2:aggregate as the project is not multi-module. The <cloverDatabase> defined in pom.xml will be used for reporting.

<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
    </configuration>
</plugin>

mvn clover2:clover

Scenario 2 - multi-module application executed on several servers independently

Assumptions:

• we have a multi-module project
  
  • like maven project with modules, or
  • ant project with several build.xml files in which <ant inheritrefs="true" inheritprops="true"> is used (so that all properties are passed)
• we deploy instrumented application (the same code) to several application servers
• tests run independently on each application server
• we're not interested in per-test coverage recording
• we want to have a single report showing combined coverage for all modules from all application servers

Overview chart:

In this scenario there are two approaches possible:

Approach #1: use one database for all modules

Approach #2: use separate database for every module but under common root

We will focus on Approach #1 as it's easier to manage. For Approach #2 - see chapter below.

Solution approach:

1) because of fact that

• we're not interested in per-test coverage AND
• we run tests independently

there is no need to configure a distributed coverage feature.

2) because of fact that

• we deploy the same code to all application servers AND
• we want to have a single report showing combined coverage from all application servers

we have the same code base, so one clover.db should be used on each application server and merging of clover databases has no sense

we should keep coverage recordings from all servers in the same location as well - note that they will not overwrite each other, because every snapshot file has unique file name

3) because of fact that

• we have a shared network drive accessible from all machines at the same absolute path
• we use single clover database for all modules

we will not need to use -Dclover.initstring at runtime, because the absolute path will be hardcoded in the instrumented code

Steps:

1) Build application with Clover

a) using Ant

<target name="all">
   <!-- Enable clover for top level module -->
   <clover-setup initstring="/path/to/network/drive/clover.db">
   <!-- Build sub-modules ensuring that properties are passed -->
   <ant inheritrefs="true" inheritprops="true" file="sub-module-a/build.xml" target="all"/>
   <ant inheritrefs="true" inheritprops="true" file="sub-module-b/build.xml" target="all"/>
</target>

b) using Maven

• define <singleCloverDatabase>true</singleCloverDatabase> in top-level pom.xml; sub-modules will inherit this setting
• define <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase> in top-level pom.xml;
• do not define <cloverDatabase> attribute in sub-modules, because it would override the singleCloverDatabase parameter

<!-- TOP LEVEL POM.XML -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
        <singleCloverDatabase>true</singleCloverDatabase>
    </configuration>
</plugin>

<!-- CHILD MODULES POM.XML -->
<!-- No need to define anything for Clover, unless you wish to have 
some module-specific settings -->

2) Deploy instrumented application to Application Servers

Copy clover.jar and your application jar/war. There's no need to copy clover.db as it's on a network drive.

3) Run tests on Application Servers

Execute your application. As we have used singleCloverDatabase and cloverDatabase pointing to absolute path on a network drive, we don't need to provide clover.initstring parameter.

4) Generate coverage report

a) using Ant

<clover-report initstring="/path/to/network/drive/clover.db">

b) using Maven

Execute clover2:clover goal. Note that you don't need to call clover2:aggregate as the project is not multi-module. The <cloverDatabase> defined in top-level pom.xml will be used for reporting.

mvn clover2:clover

Scenario 2b - multi-module application executed on several servers independently

This scenario is the same as Scenario 2, but we use separate clover.db database for every module and all of them are stored under common root.

Steps:

1) Build application with Clover

a) using Ant

This approach is generally not applicable for ant scripts.

b) using Maven

Don't define <cloverDatabase> and <singleCloverDatabase> so that default values will be used (relative path target/clover/clover.db and false).

2) Deploy instrumented application to Application Servers

No differences.

3) Run tests on Application Servers

Provide clover.initstring.basedir=/path/to/top-level-module/dir at runtime. Alternatively: use clover.initstring.prefix.

4) Generate coverage report

Before generating report you have to merge all databases. For example:

a) using Ant

<clover-merge initstring="/path/to/mergedClover.db">
   <cloverDb initstring="/path/to/network/drive/moduleA/clover.db"/>
   <cloverDb initstring="/path/to/network/drive/moduleB/clover.db"/>
</clover-merge>
<clover-report initstring="/path/to/mergedClover.db"/> 

b) using Maven

<!-- Top-level pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverMergeDatabase>/path/to/network/drive/cloverMerged.db</cloverMergeDatabase>
    </configuration>
</plugin>

mvn clover2:aggregate clover2:clover

Scenario 3 - multiple applications executed on several servers in isolation

Assumptions:

• we have multiple applications (each of them can have one or more modules)
  
  • every application has separate code base (i.e. no shared source files)
  • every application is built and instrumented separately (i.e. separate clover.db for every app)
• we deploy instrumented applications to several application servers
  
  • every application runs on it's own server (i.e. no case when two apps runs in the same JVM)
• tests run independently on each application server and for each application
• we're not interested in per-test coverage recording
• we want to have a single report showing combined coverage for all modules of all applications from all application servers

Overview chart:

Solution approach:

1) because of fact that

• we're not interested in per-test coverage AND
• we run tests independently

there is no need to configure a distributed coverage feature.

2) because of fact that

• every application has separate code base AND
• every application is built and instrumented separately AND
• we want to have a single report showing combined coverage from all application servers

it implies that

• different clover.db files should be used (one for each application on every application server where application runs)
• merging of clover databases is necessary after tests
• we should keep coverage recordings from all servers in separate locations - one location for every clover.db file

3) because of fact that

• we have a shared network drive accessible from all machines at the same absolute path
• every application runs on it's own server

we will not need to use -Dclover.initstring at runtime, because the absolute path will be hardcoded in the instrumented code

Steps:

 1) Build application with Clover

a) using Ant

Define initstring attribute for <clover-setup> or <clover-instr> tag, which will point to different directory for every application ,e.g.:

<!-- App1 build.xml -->
<clover-setup initstring="/path/to/network/drive/app1/clover.db">

<!-- App2 build.xml -->
<clover-setup initstring="/path/to/network/drive/app2/clover.db">

b) using Maven

Define <cloverDatabase> property for Clover plugin in pom.xml. You can use singleCloverDatabase in case your application is multi-module. Databases for all applications must be stored under common root (it's a limitation of clover2:merge goal; Ant clover-merge task is more flexible regarding paths). Example:

<!-- App1 pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/common-root/app1/clover.db</cloverDatabase>
    </configuration>
</plugin>

<!-- App2 pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/common-root/app2/clover.db</cloverDatabase>
        <singleCloverDatabase>true</singleCloverDatabase> <!-- assuming that app2 is multi-module -->
    </configuration>
</plugin>

mvn clean clover2:setup install

2) Deploy instrumented application to Application Servers

Copy clover.jar and your application jar/war to proper machines. There's no need to copy clover.db as it's on a network drive.

3) Run tests on Application Servers

Execute your applications. As every application runs in their own JVM and due to fact that we have used cloverDatabase (and optionally singleCloverDatabase) pointing to absolute path on a network drive, we don't need to provide clover.initstring parameter at runtime, because correct path is hardcoded in instrumented classes.

4) Generate coverage report

Before generating report you have to merge all databases. for example:

a) using Ant

<clover-merge initstring="/path/to/network/drive/cloverMerged.db">
   <cloverDb initstring="/path/to/network/drive/app1/clover.db"/>
   <cloverDb initstring="/path/to/network/drive/app2/clover.db"/>
</clover-merge>
<clover-report initstring="/path/to/cloverMerged.db"/> 

b) using Maven

<!-- Top-level pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverMergeDatabase>/path/to/network/drive/cloverMerged.db</cloverMergeDatabase>  <!-- output database -->
        <baseDir>/path/to/network/drive/common-root</baseDir>  <!-- common root -->
        <includes>*.db</includes> <!-- filename pattern, separated by comma or space -->
    </configuration>
</plugin>

mvn clover2:aggregate             # run it for multi-module applications
mvn clover2:merge clover2:clover  # run it for final report

Scenario 3b - multiple applications executed on several servers in partial isolation

This scenario is a variation of Scenario 3 in such way, that:

• location of clover.db(s) on test server is different than on build server (so you have to provide/change initstring at runtime)
• some applications are executed in the same JVM (as a consequence, you cannot pass clover.initstring as JVM argument, because you need a different value for each application).

In such case, you have to:

• instrument these applications using relative paths in <cloverDatabase> parameter (Maven) or <clover-setup initstring=""> (Ant), like below:

app1/clover.db
app2/clover.db
app2/moduleA/clover.db
app2/moduleB/clover.db

• copy generated clover.db(s) to test server, keeping their relative paths (under a common root), for instance:

/path/to/common-root/app1/clover.db
/path/to/common-root/app2/clover.db
/path/to/common-root/app2/moduleA/clover.db
/path/to/common-root/app2/moduleB/clover.db

• provide clover.iniststring.basedir=/path/to/common-root at runtime

Scenario 4 - multi-module application with distributed execution on several servers

Assumptions:

• we have a multi-module project (like Maven project with modules, or Ant project with several build.xml files with inheriting properties across ant calls)
• we deploy instrumented application (the same code) to several application servers
• unit tests are executed on one <<server>> machine, but these unit tests call business logic on one or more <<client>> machines
• we are interested in per-test coverage recording, showing combined coverage from distributed execution
• we want to have a single report showing combined coverage for all modules from all application servers

Overview chart:

Solution approach:

1) because of fact that

• we are interested in per-test coverage AND
• we run tests on several machines in parallel (so that a single test case is executed on several nodes)

it implies that

• we must configure a distributed coverage feature
• we must designate a single machine which will be our host of unit tests - see <<server>> on a diagram
• we must open network ports so that <<client>> machines will be able to connect to <<server>> on a designated port number
• we must decide how to launch <<server>> and <<clients>>, for example whether server should hold with test execution until clients are ready

2) because of fact that

• we deploy the same code to all application servers

we don't have to merge clover databases as the same database should be used on each application server

3) because of fact that

• we have a shared network drive accessible from all machines at the same absolute path

we will not need to use clover.initstring at runtime, because the absolute path will be hardcoded in the instrumented code

Steps:

1) Build application with Clover

• define how many clients will connect to JVM running unit tests;

a number greater than 0 means that server will hold until all clients are connected before it continues execution; number equal 0 means that tests will start immediately

you might have a dependency loop so that server waits for clients and clients wait for server - see below

• define server host name (default is localhost) and listening port (default is 1198)
• optionally define connection timeout (in miliseconds), retry period 

a) using Ant

<clover-setup initstring="/path/to/network/drive/clover.db">
    <distributedCoverage host="my.server.com" port="1234" numClients="2" timeout="10000"/>
</clover-setup>

b) using Maven

<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
        <singleCloverDatabase>true</singleCloverDatabase>  <!-- In case of multi-module application (optional) -->
        <distributedCoverage>
            <host>my.server.com</host>
            <port>1234</port>
            <numClients>2</numClients>
            <timeout>10000</timeout>
        </distributedCoverage>
   </configuration>
</plugin>

2) Deploy instrumented application to Application Servers

Copy clover.jar and your application jar/war to proper machines. There's no need to copy clover.db as it's on a network drive.

3) Run tests on Application Servers

On <<server>> machine

java ... -Dclover.server=true

On <<client>> machines

You don't have to provide any runtime options for JVM. They're already compiled in the code.

Potential problems

4) Generate coverage report

a) using Ant

<clover-report initstring="/path/to/network/drive/clover.db">
    <current showUniqueCoverage="true" outfile="/path/to/clover/report">
        <format type="html"/>
        <fileset dir="src"/>
    </current>
</clover-report>

b) using Maven

In order to show per-test coverage in the HTML report (showUniqueCoverage), you have to use the custom <reportDescriptor> in pom.xml and in the report descriptor set the showUniqueCoverage=true. For example:

<project name="Clover Report" default="current">
    <clover-setup initString="${cloverdb}"/>
    <target name="historical"/>
    <target name="current">
        <clover-report>
            <current showUniqueCoverage="true" outfile="${output}">  <!-- Show per-test coverage in report -->
                <format type="html"/>
            </current>
        </clover-report>
    </target>
</project>

<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
        <outputDirectory>/path/to/clover/report</outputDirectory>
        <reportDescriptor>report-descriptor.xml</reportDescriptor>    <!-- Use custom report -->
    </configuration>
</plugin>

mvn clover2:clover

Scenario 5 - multi-module application with history points

Assumptions:

• we have a multi-module project (like Maven project with modules, or Ant project with several build.xml files with inheriting properites across ant calls)
• we deploy instrumented application to one application server
• we are not interested in per-test coverage or test optimization
• we want to have a single report showing combined coverage for all modules

Overview chart:

Solution approach:

1) because of fact that we're not interested in per-test coverage or test optimization, there is no need to configure a distributed coverage feature

2) because of fact that we have one multi-module application and we want to have a single report showing combined coverage from all modules, one clover.db should be used and there's no need to merge clover databases

3) because of fact that we will generate reports with history,

• we must keep all *.xml.gz history snapshots between builds and
• we must delete clover.db and coverage files between builds

 4) because of fact that

• we have a shared network drive accessible from all machines at the same absolute path
• we use single clover database for all modules

we will not need to use clover.initstring at runtime, because the absolute path will be hard-coded in the instrumented code.

Steps:

 1) Build application with Clover

Configuration is the same as for Scenario 2.

a) using Ant

<target name="all">
   <!-- Enable clover for top level module -->
   <clover-setup initstring="/path/to/network/drive/clover.db">
   <!-- Build sub-modules ensuring that properties are passed -->
   <ant inheritrefs="true" inheritprops="true" file="sub-module-a/build.xml" target="all"/>
   <ant inheritrefs="true" inheritprops="true" file="sub-module-b/build.xml" target="all"/>
</target>

b) using Maven

<!-- TOP LEVEL POM.XML -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
        <singleCloverDatabase>true</singleCloverDatabase>
    </configuration>
</plugin>

<!-- CHILD MODULES POM.XML -->
<!-- No need to define anything for Clover, unless you wish to have some module-specific settings -->

mvn clover2:setup test 

2) Deploy instrumented application to Application Servers

Remove previous version of application and copy clover.jar and your application jar/war. There's no need to copy clover.db as it's on a network drive.

3) Run tests on Application Servers

Execute your application. As we have used singleCloverDatabase and cloverDatabase pointing to absolute path on a network drive, we don't need to provide clover.initstring parameter at runtime.

4) Generate coverage report

a) using Ant

<clover-report initstring="/path/to/network/drive/clover.db"> 
    <current outfile="/path/to/clover/report/current" title="Coverage Report">
        <format type="html"/>
        <fileset dir="src"/>
    </current>
    <historical outfile="/path/to/clover/report/historical" title="Historical Report" historyDir="/path/to/clover/historypoints">
       <format type="html"/>
    </historical>
</clover-report>

<clover-historypoint historyDir="/path/to/clover/historypoints">
       <fileset dir="src"/>
</clover-historypoint>

b) using Maven

<!-- Top-level pom.xml -->
<plugin>
    <groupId>com.atlassian.maven.plugins</groupId>
    <artifactId>maven-clover2-plugin</artifactId>
    <configuration>
        <cloverDatabase>/path/to/network/drive/clover.db</cloverDatabase>
        <singleCloverDatabase>true</singleCloverDatabase>
        <generateHistorical>true</generateHistorical>
        <generateHtml>true</generateHtml>
        <historyDir>/path/to/clover/historypoints</historyDir>
        <outputDirectory>/path/to/clover/report</outputDirectory>
    </configuration>
</plugin>

mvn clover2:clover clover2:save-history

Decision matrix

Instrumentation of source code

Application execution

Environment configuration

Per-test coverage and test optimization vs distributed coverage

References

• About Distributed Per-Test Coverage
• Working with Distributed Applications
  
  • Using Distributed Per-test Coverage with Clover-for-Ant
  • Using Distributed Per-test Coverage with Maven 2 and 3
• Using Clover for web applications
• FAQ / Atlassian Answers
  
  • When using Clover, why do I get a java.lang.NoClassDefFoundError when I run my code?
  • No coverage being generated from WebLogic hosted instrumented EAR
  • Clover on J2EE: Manual Testing