Jira Service Management 5.4.x Long Term Support release performance report

This page compares the performance of Jira Service Management 4.20 with Jira Service Management 5.4 Long Term Support release.

About Long Term Support releases

We recommend upgrading Jira Service Management regularly, but if your organization's process means you only upgrade about once a year, upgrading to a Long Term Support release is a good option. It provides continued access to critical security, stability, data integrity and performance issues until this version reaches end of life.

Performance

As with all Long Term Support releases, we aim to provide the same, if not better, performance. Jira Service Management 5.4 testing demonstrates significant performance improvements in viewing workload reports and Assets import speed and scaling. We also saw notable improvements in viewing the customers page, viewing service issues, and creating a customer request.

Some performance regressions were seen in viewing requests, viewing queues and Assets object creation. These can be attributed to functional improvements made in the product since Jira Service Management 4.20 LTS.

In this section, we’ll compare Jira Service Management 4.20 to Jira Service Management 5.4, for both Server and Data Center. We ran the same extensive test scenario for both versions, divided into four categories:

• Lighter actions

• Medium actions

• Heavy actions (that take longer to run)

• Insight actions (related to the asset management functionality)

The performance was measured under a user load we estimate to be peak traffic, on an instance with 5000 users. For more details on actions and how we tested them, see Testing methodology .

Lighter actions

The graph shows differences in response times of individual actions. You can view the data used to build the graph below.

Medium actions

The graph shows differences in response times of individual actions. You can view the data used to build the graph below.

Heavy actions

The graph shows differences in response times of individual actions. You can view the data used to build the graph below.

Assets actions

The graph shows differences in response times of individual actions. You can view the data used to build the graph below.

Assets import speed and scaling

In addition to the performance of user operations in the product, we also made improvements to the way Assets uses memory, performs imports, and handles bulk operations.

• Import object creation is now 55-70% faster.

• Large datasets now require 20-35% less memory.

• Assets reindexing is now 10x faster.

• Querying and importing complex object type hierarchies is now up to 25x faster*.

• Bulk deleting objects via the Assets user interface is now 4.8x faster.

• Offloading imports to the disk rather than the memory means that Assets now supports much larger imports than previously.

^{* Demonstrated with National Vulnerability Database (NVD) import speed}

In summary

The performance is largely stable across the product and under high load, with a few improvements. The highlights:

• Viewing workload reports (medium) is now 19.4x faster.

• Viewing the customers page is now 1.1x faster.

• Viewing service issues is now 1.2x faster.

• Creating a customer request is now 1.1x faster.

• Import object creation is now 55-70% faster.

• Large datasets now require 20-35% less memory.

Some performance regressions were seen in viewing requests, viewing queues and Assets object creation. These can be attributed to new request list functionality (4.21 and 5.3), changes made to improve SLA calculation accuracy (5.0), and various accessibility and user interface improvements in Assets (5.0, 5.1, 5.2, 5.3).

We will continue to improve future performance so that service teams can move with ease through their workspace, and our largest customers can scale confidently.

Testing methodology

The following sections detail the testing environment, including hardware specification, and methodology we used in our performance tests.

How we tested

Before we started testing, we needed to determine what size and shape of dataset represents a typical large Jira Service Management instance. To achieve that, we used our Analytics data to form a picture of our customers' environments and what difficulties they face when scaling Jira Service Management in a large organization.

We’ve also included a dataset for Assets, since it’s part of Jira Service Management Data Center.

The following table presents the rounded values of the 99th percentile of each data dimension. We used these values to generate a sample dataset with random test data.

Baseline data set

Assets data set

Actions performed

We chose a mix of actions that would represent a sample of the most common user actions. An action in this context is a complete user operation, like opening an issue in the browser window. The following table details the actions that we included in the script, for our testing persona, indicating how many times each action is repeated during a single test run.

Lighter, medium, and heavy actions

Assets actions

Test environment for user actions

The performance tests were all run on a set of AWS EC2 instances. For each test, the entire environment was reset and rebuilt, and then each test started with some idle cycles to warm up instance caches. Below, you can check the details of the environments used for Jira Service Management Server and Data Center, as well as the specifications of the EC2 instances.

To run the tests, we used 21 scripted browsers and measured the time taken to perform the actions. Each browser was scripted to perform a random action from a predefined list of actions and immediately move on to the next action (ie. zero think time). Note that this resulted in each browser performing substantially more tasks than would be possible by a real user, and you should not equate the number of browsers to represent the number of real-world concurrent users.

Each test was run for 40 minutes, after which statistics were collected.

Here are the details of our test environment: