Jira Service Management 4.20.x Long Term Support release performance report

This page compares the performance of Jira Service Management 4.13 (formerly Jira Service Desk) with Jira Service Management 4.20 Long Term Support release.

About Long Term Support releases

We recommend upgrading Jira Service Management regularly, but if your organisation's process means you only upgrade about once a year, upgrading to a Long Term Support release is a good option. It provides continued access to critical security, stability, data integrity and performance issues until this version reaches end of life.

Performance

As with all Long Term Support releases, we aim to provide the same, if not better, performance. Jira Service Management 4.20 testing demonstrates significant performance improvements in viewing queues, viewing workload reports, and inviting team members. This release also includes key Insight asset management benchmarking for the first time since bundling the app in Jira Service Management Data Center 4.15.

In this section, we’ll compare Jira Service Management 4.13 to Jira Service Management 4.20, for both Server and Data Center. We ran the same extensive test scenario for both versions, divided into four categories:

• Lighter actions

• Medium actions

• Heavy actions (that take longer to run)

• Insight actions (related to the asset management functionality)

The performance was measured under a user load we estimate to be peak traffic, on a 5000 users instance. For more details on actions and how we tested them, see Testing methodology.

Lighter actions

Difference between individual actions. You can view the data used to build the graph below it.

Medium actions

Difference between individual actions. You can view the data used to build the graph below it.

Heavy actions

Difference between individual actions. You can view the data used to build the graph below it.

Insight actions

Difference between individual actions. You can view the data used to build the graph below it.

In summary

The performance is stable across the product, under high load, with a few improvements. The highlights:

• Viewing queues is now 2.2x faster

• Viewing workload reports (medium) is now 2x faster 

• Inviting team members is now 1.7x faster

A notable exception is a slight 300ms degradation in "View organizations page". 

We'll continue to invest in improving future performance so that service desk teams can move with ease through their workspace, and our largest customers can scale confidently.

Testing methodology

The following sections detail the testing environment, including hardware specification, and methodology we used in our performance tests.

How we tested

Before we started the test, we needed to determine what size and shape of dataset represents a typical large Jira Service Management instance. To achieve that, we used our Analytics data to form a picture of our customers' environments and what difficulties they face when scaling Jira Service Management in a large organization.

We’ve also included a dataset for Insight, since it’s now part of Jira Service Management Data Center.

The following table presents the rounded values of the 99th of each data dimension. We used these values to generate a sample dataset with random test data.

Baseline data set

Insight data set

Actions performed

We chose a mix of actions that would represent a sample of the most common user actions. An action in this context is a complete user operation, like opening an issue in the browser window. The following table details the actions that we included in the script, for our testing persona, indicating how many times each action is repeated during a single test run.

Lighter, medium, and heavy actions

Insight actions

Test environment for user actions

The performance tests were all run on a set of AWS EC2 instances. For each test, the entire environment was reset and rebuilt, and then each test started with some idle cycles to warm up instance caches. Below, you can check the details of the environments used for Jira Service Desk Server and Data Center, as well as the specifications of the EC2 instances.

To run the tests, we used 21 scripted browsers and measured the time taken to perform the actions. Each browser was scripted to perform a random action from a predefined list of actions and immediately move on to the next action (ie. zero think time). Please note that it resulted in each browser performing substantially more tasks than would be possible by a real user, and you should not equate the number of browsers to represent the number of real-world concurrent users.

Each test was run for 40 minutes, after which statistics were collected.

Here are the details of our test environment:

Test environment for indexing measures