Performance Testing Scripts

Setup

You will need the following -

• A Confluence server, set up and running with an admin user. The scripts assume a default username and password for this user: 'admin'/'admin'.
• Ensure the Confluence Remote API is enabled in the administration options. See Enabling the Remote API for details on how to configure this.
• Apache JMeter
• The load testing scripts and resources which are available in our public Maven repository— Please choose the version that most closely matches your Confluence version and download the ZIP or Gzip file in that directory. If in doubt, download the ZIP file archive.

  • e.g. 

    Confluence Version	Performance Test Script
    4.0.3 - 4.1.x	4.0.3
    4.2 - 5.0.x	4.2.3
    5.1 - 5.4.x	4.3

The test scripts have been updated to work with Confluence 3.4 in version 3.4.  Using an older version of the tests will result in errors when running the test.

Quick, Just Tell Me How To Run It.

If you don't want to read the rest of this document, here are the main points:

1. Download and Unzip the performance tests
2. Open a command prompt and change directory to the performanceTest directory that has just been unzipped.

3. Create the test data:

   <jmeter location>/bin/jmeter -n -t jmeter-test-setup.jmx -Jspace.zip=<path to a demo space ZIP file> -Jadmin.user=<username> -Jadmin.pass=<password>
   

4. Run the test:

   <jmeter location>/bin/jmeter -n -t jmeter-test-fixedload.jmx
   

The remainder of this document is just an elaboration of those two steps.

Creating the Test Data

A known data set is required to run the testing against. By default this is the Confluence demo space (space key = DS) although this can be changed (more on this later). If you decide to use the Confluence demo space, ensure that the group "confluence-users" is able to update content in this space.

The script jmeter-test-setup.jmx is used to:

• create a set of users to be used in the test
• import the Confluence demo space for running tests against.

You should first ensure that you don't already have the demo space (key = DS) on your test instance. Delete it if you do.

Run the script from the performanceTest directory as follows:

<jmeter location>/bin/jmeter -n -t jmeter-test-setup.jmx -Jspace.zip=<path to a space export.zip>-Jadmin.user=<username> -Jadmin.pass=<password>

Where:

• <path to a space export.zip> is the absolute path to the space export zip you want to be used in your testing. For example, the path to demo-site.zip as found in your Confluence distribution or source: <confluence install>/confluence/WEB-INF/classes/com/atlassian/confluence/setup/demo-site.zip

• <username> and <password> are the username and password for an admin user that is able to create Confluence users and to import spaces.

By default the setup process will create 250 users — 50 each of the following formats: tstreader<n>, tstcommentor<n>, tsteditor<n>, tstcreator<n> and tstsearcher<n>. The password for each matches the username.

A typical run of the setup script will only take a few seconds.

Removing the Test Data

You can reverse the effects of the setup script by setting the remove.data parameter to true, e.g.

<jmeter location>/bin/jmeter -n -t jmeter-test-setup.jmx -Jremove.data=true -Jadmin.user=<username> -Jadmin.pass=<password>

Setup Script Parameters

You can modify the behaviour of the setup script via JMeter parameters. These are supplied on the command line in the form -J<parameter name>=<parameter value>.

Setup Script Output

On the console you will see no obvious indication of success or otherwise. JMeter will output something similar to this:

Created the tree successfully
Starting the test @ Mon Apr 14 17:35:08 EST 2008 (1208158508222)
Tidying up ... @ Mon Apr 14 17:35:08 EST 2008 (1208158508928)
... end of run

The scripts location/results directory will contain the file jmeter-result-setuptest.jtl. There were failures or errors if there are any assertions in this file that have the value true for the failure or error element, e.g.

<assertionResult>
<name>Manage Users</name>
<failure>true</failure>
<error>false</error>
<failureMessage>Test failed: URL expected to contain /browseusers.action/</failureMessage>
</assertionResult>

Running the Test

The test script itself will put Confluence under a fixed load. Each thread group will attempt to do a certain amount of work for a prescribed period of time (30 minutes by default). This is by design so that load during test runs can accurately be compared against each other.

Execute the test as follows:

<jmeter location>/bin/jmeter -n -t jmeter-test-fixedload.jmx

Where:
<scripts location> is the absolute path to where you extracted the scripts e.g. /Users/YourName/Download/performanceTest. This is needed for the script to find its external resources.

Test Behaviour

The test has a number of parameters to tweak its behaviour but generally speaking it has the rough format of:

• 5 groups of users - readers, commentors, searchers, editors and creators.
  • readers simply view a set of individual pages or browse space functionality.
  • commentors add comments to a set of pages.
  • searchers perform searches on a fixed set of keywords.
  • editors make small additions to the end of a set of pages.
  • creators add new pages to a particular space.
• Each individual user in each group will repeat for a fixed amount of time with a small pause between each request.

Note that there is no execution of JavaScript by the client. Keep this in mind if you use this test to gauge Confluence performance in a production environment.

There is also very little use of permissions in these tests. All data involved is accessible to all of the test users.

Test Script Parameters

You can modify the behaviour of the test script via JMeter parameters. These are supplied on the command line in the form -J<parameter name>=<parameter value>.

Test Thread Parameters

So with the default parameters, you are emulating a load on Confluence of 33 concurrent users who will each be hitting the server approximately every 2 seconds (16 users per second).

23 of these users are read only (searchers or readers) and 10 of them are read/write — 11 read only users per second and 5 read/write users per second.

Test Script Output

During the run of the test script Jmeter will output progress to the console of the form:

Created the tree successfully
Starting the test @ Fri Apr 18 00:07:39 EST 2008 (1208441259523)
Display Summary Results During Run + 462 in 77.6s = 5.9/s Avg: 1564 Min: 18 Max: 33738 Err: 1 (0.22%)
Display Summary Results During Run + 1338 in 189.9s = 7.0/s Avg: 3596 Min: 24 Max: 34545 Err: 0 (0.00%)
Display Summary Results During Run = 1800 in 257.6s = 7.0/s Avg: 3074 Min: 18 Max: 34545 Err: 1 (0.06%)
Display Summary Results During Run + 1046 in 200.9s = 5.2/s Avg: 4529 Min: 40 Max: 50461 Err: 0 (0.00%)
Display Summary Results During Run = 2846 in 438.2s = 6.5/s Avg: 3609 Min: 18 Max: 50461 Err: 1 (0.04%)
Display Summary Results During Run + 677 in 201.2s = 3.4/s Avg: 6638 Min: 46 Max: 27636 Err: 0 (0.00%)
Display Summary Results During Run = 3523 in 618.1s = 5.7/s Avg: 4191 Min: 18 Max: 50461 Err: 1 (0.03%)
Display Summary Results During Run + 561 in 197.5s = 2.8/s Avg: 8326 Min: 171 Max: 39494 Err: 0 (0.00%)
Display Summary Results During Run = 4084 in 798.3s = 5.1/s Avg: 4759 Min: 18 Max: 50461 Err: 1 (0.02%)
Display Summary Results During Run + 555 in 199.2s = 2.8/s Avg: 8247 Min: 160 Max: 45270 Err: 0 (0.00%)
Display Summary Results During Run = 4639 in 978.0s = 4.7/s Avg: 5177 Min: 18 Max: 504