Testing disk access speed for a Java application

Problem

The Atlassian Server application appears to be experiencing performance issues and is running slowly. These applications could include Java based software such as JIRA, Confluence, Bitbucket Server, Bamboo and FishEye/Crucible.

For example in JIRA, this will be particularly noticeable in the following areas:

  • Reindexing
  • Searching
  • The Issue Navigator
  • Gadgets
  • Opening Issues

Diagnosis

Disk access speed is critical for the Atlassian applications performance.

If the application is running slowly, disk speed can be a potential root cause. It's possible to isolate that cause using this Support Tools Utility to benchmark the disk access speed.

  1. Download the Support Tools Utility.

    (warning)This is a file with a .jar extension and sometimes Internet Explorer renames these to .zip. If this is the case, rename the file to .jar; do not unzip the file.

  2. Open a terminal window (in Windows, go to Start >> Run >> type in 'cmd').
  3. When running the tool, write access to the temporary directory is required - it will create and delete a large number of files in order to calculate the disk speed.
  4. Run the Support Tools Utility with the following:

    java -Djava.io.tmpdir=<index directory> -jar support-tools.jar
    (info) For JIRA make sure to test against the default index directory <jira_home>/caches/indexes. For FishEye/Crucible, the test should be run against <FishEye installdirectory>/cache and <FishEye installdirectory>/var/cache.
  5. Pick the average of the results to assess and review our Grading the Results section for further info on how to determine the results of the speed tests.

For example in JIRA, disk access performance is particularly important due to the heavy use of disk I/O to the indexes, because they are read each time a search is made and written each time an update to an issue is committed (see Searching, Indexing, and filters troubleshooting for further information).

Operation

This basic test will perform the open, read/write, close and delete operations 1000 times to a temporary file in the java.io.tmpdir directory for each measurement, reporting the results as in the assessment section below. This test is the closest we can get to simulating the performance of a Java process when it comes to accessing the file system, allowing it to be an accurate measurement for the application performance when utilising the disk. Other disk measurement tools may provide different results if they do not measure the specific performance of the Java JDK/JRE.

The Java source can be found within the JAR - if the file is uncompressed it can be viewed.

Assessment

This tool has been designed primarily to perform basic disk assessment of Java running on the operating system, not the actual speed of the disk itself. It is highly possible a very fast disk can run Java slowly due to OS-level issues. This is not a substitute for a full disk analysis software suite and if the disk appears to be experiencing hardware issues or is slow, please raise this with the System Administrator or hosting provider of the server.

When reviewing the benchmark results:

  • Focus primarily on the average result.
  • When the open & close speeds are bad and the r/w is good, the likely cause is anti-virus.
  • A high minimum and average speed may be indicative of a low speed disk or hardware fault.
  • If a disk has a high average and low minimum speed, there may be an environmental factor at play (see below for further information).
  • For JIRA open and close are important values because the indexes are accessed very often - the average search will close a file many times.

Grading the Results 

Below is a breakdown of approximate average speeds that can be used to grade the results.

Statistic Excellent Good Bad
Open < 40,000 < 80,000 > 150,000
Read/Write < 40,000 < 60,000 > 100,000
Close < 20,000 < 40,000 > 100,000
Delete < 50,000 < 200,000 > 300,000

Possible Causes

Examples of environmental factors that can cause slow disk access are as follows:

  • Anti-Virus software. As in our Crashes and Performance Issues Troubleshooting KB article, Anti-Virus software can be detrimental to the operation of the applicationa as it will limit open and close times.
  • A remote disk or shared drive.
  • Synchronisation to another machine over a slow network.
  • A virtualised OS, such as VMWare (see Virtualizing JIRA (JIRA on VMware) for further information).
  • A disk defragment job may be running.
  • Hardware issues such as disk failure.
  • File system encryption.
  • Automated compression of files controlled by the OS.
  • Specific issues with the Java version and OS. This is rare occurrence, however a bug or known issue within the JVM may cause it to perform poorly on a specific OS.
  • Other applications or operations that are currently using the disk.
  • The disk capacity may be nearing full, which on some OS can slow the performance of the disk (in this particular example, it was on Solaris).

Sample Benchmarks

We have included some sample benchmarks below.

Excellent
TOTALS
----    ----    ----    ----    ----
stat    avg     median  min     max
----    ----    ----    ----    ----
open    35,289  36,000  29,000  472,000
r/w     30,934  31,000  23,000  77,000
close   14,988  5,000   5,000   9,796,000
delete  39,135  40,000  31,000  123,000
----    ----    ----    ----    ----
All times are in nanoseconds.

This is a premier hosting provider using disk hardware technology equivalent to a SSD.

Good
TOTALS
----    ----    ----    ----    ----
stat    avg     median  min     max
----    ----    ----    ----    ----
open    75,830  62,000  44,000  2,869,000
r/w     46,407  40,000  35,000  907,000
close   5,751   5,000   4,000   336,000
delete  118,942 81,000  65,000  22,864,000
----    ----    ----    ----    ----
All times are in nanoseconds.
Bad
TOTALS
----    ----    	----    	----    	----
stat    avg     	median  	min     	max
----    ----    	----    	----    	----
open    275,886 	267,184 	246,412 	2,544,101
r/w     92,956  	81,538  	70,727  	7,358,896
close   3,194,958   3,177,771   3,076,064   4,331,872
delete  542,650 	530,271 	514,226 	1,771,416
----    ----    	----    	----    	----
All times are in nanoseconds.

This benchmark is of very fast disk with a virus scanner enabled.

Average
TOTALS
----	----	----	----	----
stat	avg		median	min		max
----	----	----	----	----
open	133,704	121,000	83,000	12,787,000
r/w		30,892	26,000	22,000	3,176,000
close	40,652	40,000	10,000	108,000
delete	274,710	254,000	211,000	15,151,000
----	----	----	----	----
All times are in nanoseconds.

This is a WD Black 3.0GB/s 7.2k RPM drive running 5 JIRA application instances during low load (whilst we would not ever recommend to do this, it does highlight the impact of multiple applications running on the same server).

Was this helpful?

Thanks for your feedback!

Why was this unhelpful?

Have a question about this article?

See questions about this article

Powered by Confluence and Scroll Viewport