Test disk access speed for a Java application

Atlassian Knowledge Base

On this page

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Problem

The Atlassian Server application appears to be experiencing performance issues and is running slowly. These applications could include Java based software such as JIRA, Confluence, Bamboo, and Fisheye/Crucible.

For example in JIRA, this will be particularly noticeable in the following areas:

  • Reindexing
  • Searching
  • The Issue Navigator
  • Gadgets
  • Opening Issues


If you are running Bitbucket datacenter, please refer to the article below for details on disk benchmarking.

Testing NFS disk access speed for Bitbucket Data Center and git operations


Diagnosis

Disk access speed is critical for the Atlassian applications performance.

If the application is running slowly, disk speed can be a potential root cause. It's possible to isolate that cause using this Support Tools Utility to benchmark the disk access speed.


Download the Support Tools Utility to any writable folder on the actual JIRA server (for example ~/$username$/temp on Linux or c:\temp on Windows).

This is a file with a .jar extension and sometimes Internet Explorer renames these to .zip. If this is the case, rename the file to .jar; do not unzip the file.

  1. For Windows: Open a terminal window (go to Start >> Run >> type in 'cmd').
  2. For Linux/Unix: Open a command line.
  3. When running the tool, write access to the temporary directory is required - it will create and delete a large number of files in order to calculate the disk speed.
  4. CD to the directory where the tools was downloaded and run it with the following:

    java -Djava.io.tmpdir=<index directory> -jar support-tools.jar

     Depending on your application please test the following directories using the <index directory> path below:

    • For JIRA make sure to test against the default index directory <jira_home>/caches/indexes on JIRA 7.x and below, <jira_home>/caches/indexesV1 folder on JIRA 8.x and above. Notice if you have Insight - run the check against <jira_home>/caches/insight_indexes too.

    • For Confluence make sure to test against <confluence-home>/index.

    • For Fisheye/Crucible, the test should be run against <FISHEYE_INST>/cache and <FISHEYE_INST>/var/cache.

    If you're running Data Center version of JIRA or Confluence - please run the speed check on each node as the indexes are stored locally on each node

  5. Pick the average of the results to assess and review our Grading the Results section for further info on how to determine the results of the speed tests. 
    For example in JIRA, disk access performance is particularly important due to the heavy use of disk I/O to the indexes, because they are read each time a search is made and written each time an update to an issue is committed (see Troubleshoot index problems in Jira server for further information).

Operation

This basic test will perform the open, read/write, close, and delete operations 1000 times to a temporary file in the java.io.tmpdir directory for each measurement, reporting the results as in the assessment section below. This test is the closest we can get to simulating the performance of a Java process when it comes to accessing the file system, allowing it to be an accurate measurement for the application performance when utilizing the disk. Other disk measurement tools may provide different results if they do not measure the specific performance of the Java JDK/JRE.

The Java source can be found within the JAR - if the file is uncompressed it can be viewed.

Resolution

This tool has been designed primarily to perform basic disk assessment of Java running on the operating system, not the actual speed of the disk itself. It is highly possible a very fast disk can run Java slowly due to OS-level issues. This is not a substitute for a full disk analysis software suite and if the disk appears to be experiencing hardware issues or is slow, please raise this with the System Administrator or hosting provider of the server.

When reviewing the benchmark results:

  • Focus primarily on the average result.
  • When the open & close speeds are bad and the r/w is good, the likely cause is anti-virus.
  • A high minimum and average speed may be indicative of a low speed disk or hardware fault.
  • If a disk has a high average and low minimum speed, there may be an environmental factor at play (see below for further information).
  • For JIRA open and close are important values because the indexes are accessed very often - the average search will close a file many times.

Grading the Results 

Below is a breakdown of approximate average speeds that can be used to grade the results.


StatisticExcellentOKBad
Open< 40,00040,000 - 150,000> 150,000
Read/Write< 40,00040,000 - 100,000> 100,000
Close< 20,00020,000 - 100,000> 100,000
Delete< 50,00050,000 - 300,000> 300,000

Possible Causes

Examples of environmental factors that can cause slow disk access are as follows:

  • Anti-Virus software. As in our Common causes for Jira Server crashes and performance issues KB article, Anti-Virus software can be detrimental to the operation of the application as it will limit open and close times.
  • A remote disk or shared drive.
  • Synchronization to another machine over a slow network.
  • A virtualized OS, such as VMWare (see Run Jira server in a virtualized environment for further information).
  • A disk defragments job may be running.
  • Hardware issues such as disk failure.
  • File system encryption.
  • Automated compression of files controlled by the OS.
  • Specific issues with the Java version and OS. This is a rare occurrence, however, a bug or known issue within the JVM may cause it to perform poorly on a specific OS.
  • Other applications or operations that are currently using the disk.
  • The disk capacity may be nearing full, which on some OS can slow the performance of the disk (in this particular example, it was on Solaris).
  • Lucene has some issues with file deletion in Windows.
    Since Lucene uses mmapdirectory and specifically to the Windows OS, it keeps the handles open to these mapped files even after close, not allowing these files to be deleted. Lucene Class MMapDirectory

Sample Benchmarks*

We have included some sample benchmarks below.

Excellent
TOTALS
----    ----    ----    ----    ----
stat    avg     median  min     max
----    ----    ----    ----    ----
open    35,289  36,000  29,000  472,000
r/w     30,934  31,000  23,000  77,000
close   14,988  5,000   5,000   9,796,000
delete  39,135  40,000  31,000  123,000
----    ----    ----    ----    ----
All times are in nanoseconds.

This is a premier hosting provider using disk hardware technology equivalent to an SSD.

OK
TOTALS
----    ----    ----    ----    ----
stat    avg     median  min     max
----    ----    ----    ----    ----
open    75,830  62,000  44,000  2,869,000
r/w     46,407  40,000  35,000  907,000
close   5,751   5,000   4,000   336,000
delete  118,942 81,000  65,000  22,864,000
----    ----    ----    ----    ----
All times are in nanoseconds.
Bad
TOTALS
----    ----    	----    	----    	----
stat    avg     	median  	min     	max
----    ----    	----    	----    	----
open    275,886 	267,184 	246,412 	2,544,101
r/w     92,956  	81,538  	70,727  	7,358,896
close   3,194,958   3,177,771   3,076,064   4,331,872
delete  542,650 	530,271 	514,226 	1,771,416
----    ----    	----    	----    	----
All times are in nanoseconds.

This benchmark is of very fast disk with a virus scanner enabled.

Average
TOTALS
----	----	----	----	----
stat	avg		median	min		max
----	----	----	----	----
open	133,704	121,000	83,000	12,787,000
r/w		30,892	26,000	22,000	3,176,000
close	40,652	40,000	10,000	108,000
delete	274,710	254,000	211,000	15,151,000
----	----	----	----	----
All times are in nanoseconds.



This is a WD Black 3.0GB/s 7.2k RPM drive running 5 JIRA application instances during low load (whilst we would not ever recommend to do this, it does highlight the impact of multiple applications running on the same server).

*Notice. The above results are given as a guideline for local <index directory> folder only. There is no need to perform these tests on $SHARED_HOME in Data Center configurations as the indexes are stored locally on each node and not on the SHARED drive. Sometimes support may request to run the same tests on the $SHARED_HOME, but that's only if performance issues relevant to attachments or index recovery mechanism (such as when a node starts up after a prolonged period of downtime and if the index recovery is enabled in the respective application). On $SHARED_HOME for an Atlassian public instance we have the following values: 


TOTALS
---- ---- ---- ---- ----
stat avg median min max
---- ---- ---- ---- ----
open 4,790,174 4,638,824 3,943,820 13,139,894
r/w 14,947 13,312 3,519 83,294
close 4,398,125 4,297,768 3,708,446 16,138,393
delete 4,323,586 4,178,885 3,581,537 33,963,856
---- ---- ---- ---- ----
All times are in nanoseconds.

(!) NOTE: These are strictly for reference, not recommended nor performance target values.



Description

Disk access speed is critical for the Atlassian applications performance.

If the application is running slowly, disk speed can be a potential root cause. It's possible to isolate that cause using this Support Tools Utility to benchmark the disk access speed.

ProductJIRA, Confluence, Bamboo, Fisheye, Crucible
PlatformServer, Data Center
Last modified on Sep 23, 2022

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.