Generating a thread dump

System administration

On this page

Related content

Still need help?

The Atlassian Community is here for you.

Ask the community

Occasionally, Jira may appear to 'freeze' during execution of an operation. During these times, it is helpful to retrieve a thread dump — a log containing information about currently running threads and processes within the Java Virtual Machine. Taking thread-dumps is a non-destructive process that can be run on live systems. This document describes the steps necessary to retrieve a thread dump.

The steps necessary to retrieve the thread dump are dependant on the operating system Jira is running in — please follow the appropriate steps below.

Windows environment

Jira running from startup.bat

  1. In the Command Console window where Jira is running, open the properties dialog box by right clicking on the title bar and select "Properties".
  2. Select the Layout tab.
  3. Under Screen Buffer Size, set the Height to 3000.
  4. Click OK.
  5. With the same command console in focus, press CTRL-BREAK. This will output the thread dump to the command console.
  6. Scroll back in the command console until you reach the line containing "Full thread dump".
  7. Right click the title bar and select Edit -> Mark. Highlight the entire text of the thread dump.
  8. Right click the title bar and select Edit -> Copy. The thread dump can then be pasted into a text file.
     

Jira running as a Windows service

Using jstack

The JDK ships with a tool named jstack for generating thread dumps.

  1. Identify the process. Launch the task manager by, pressing Ctrl + Shift + Esc and find the Process ID of the Java (Jira) process. You may need to add the PID column using View -> Select Columns ...

  2. Run jstack to capture a single thread dump or multiple thread dumps at set intervals:

    Capture a single thread dump...

    Use the following command to capture a single thread dump of the process ID <JIRA_PID>:

    jstack.exe -l <JIRA_PID> > threaddump.txt

    For example, if the PID is 22668, enter:

    jstack.exe -l 22668 > threaddump.txt

    This will output to threaddump.txt in your current directory.

    Capture multiple thread dumps at set intervals...

    Use the following command to capture 6 thread dumps of the process id <JIRA_PID> in 10-second intervals between each thread dump:

    for /L %n in (1,1,6) do timeout 10 | jstack.exe -l <JIRA_PID> > threaddump-%n.txt

    For example, if the PID is 22668, enter:

    for /L %n in (1,1,6) do timeout 10 | jstack.exe -l 22668 > threaddump-%n.txt

    This will output to threaddump-%n.txt, where %n is the number of the loop iteration (starting at 1).

    You can modify the timeout command parameter to adjust the time between each thread dump and the range in the in (start,step,end) clause to adjust the number of thread dumps to capture. The (1,1,6) range in the example above means the following:

    • Start at 1
    • Increment by 1
    • End at 6

    Common issues with jstack:

    • You must run jstack as the same user that is running Jira.

    • If you get the error "Not enough storage is available to process this command", download the 'psexec' utility from here, and then run one of the following commands, where <JIRA_PID> is the Jira process ID (for example, 22668):

      • To capture a single thread dump:

        jstack.exe -l <JIRA_PID> > threaddump.txt

      • To capture multiple thread dumps at set intervals:

        for /L %n in (1,1,6) do timeout 10 | psexec -s jstack.exe -l $JIRA_PID > threaddump-%n.txt
    • If the jstack executable is not in your $PATH, then look for it in your <JDK_HOME>/bin directory
    • If you receive java.lang.NoClassDefFoundError: sun/tools/jstack/JStack check that tools.jar is present in your JDK's lib directory. If it is not, download a full version of the JDK.

Linux/Unix/OS X environment

Linux/Unix command line

  1. Identify the java process that Jira is running in. This can be achieved by running a command similar to:

    ps -ef | grep java

    The process will appear similarly as follows:

    keithb     910   873  1 17:01 pts/3    00:00:18 /usr/java/jdk/bin/java -Xms128m -Xmx256m
-Xms128m -Xmx256m -Djava.awt.headless=true -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.awt.headless=true -Djava.endorsed.dirs=/tmp/atlassian-jira-enterprise-3.6-standalone/common/endorsed
-classpath :

  2. In order to retrieve the thread dump, execute the command

    kill -3 <pid>

    where pid is the process id — in this case, 910.

  3. The thread dump will be written to the Tomcat console output. The console output is redirected to the logs/catalina.out file, which can be found in the Jira application installation directory for JIRA Standalone / Installer.

Linux/Unix Alternative: Generating thread dumps using jstack

If you have trouble using kill -3 <pid> to obtain a thread dump, try using jstack a java utility that will output stack traces of Java threads for a given process.

  1. Identify the java process that Jira is running in. This can be achieved by running a command similar to:

    ps -ef | grep java

    The process will appear similarly as follows:

    adam 22668 0.3 14.9 1691788 903928 ? Sl Jan27 9:36 /usr/lib/jvm/java-6-sun-1.6.0.14/bin/java -Djava.util.logging.config.file=/home/adam/Products/installs/atlassian-jira-enterprise-4.0.1-standalone/conf/logging.properties -XX:MaxPermSize=256m -Xms128m -Xmx1048m -Djava.awt.headless=true -Datlassian.standalone=JIRA -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Dmail.mime.decodeparameters=true -Datlassian.mail.senddisabled=false -Datlassian.mail.fetchdisabled=false -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/home/adam/Products/installs/atlassian-jira-enterprise-4.0.1-standalone/common/endorsed -classpath /home/adam/Products/installs/atlassian-jira-enterprise-4.0.1-standalone/bin/bootstrap.jar -Dcatalina.base=/home/adam/Products/installs/atlassian-jira-enterprise-4.0.1-standalone -Dcatalina.home=/home/adam/Products/installs/atlassian-jira-enterprise-4.0.1-standalone -Djava.io.tmpdir=/home/adam/Products/installs/atlassian-jira-enterprise-4.0.1-standalone/temp org.apache.catalina.startup.Bootstrap start

  2. Run jstack to capture a single thread dump or multiple thread dumps at set intervals:

    Capture a single thread dump...

    Use the following command to capture a single thread dump of the process ID <JIRA_PID>:

    jstack <JIRA_PID> > threaddump.txt

    For example, if the PID is 22668, enter:

    jstack 22668 > threaddump.txt

    The output will be saved to threaddump.txt in the current directory.

    Capture multiple thread dumps at set intervals...

    Use the following command to capture 6 thread dumps of the process ID <JIRA_PID> in 10-second intervals between each thread dump:

    for i in $(seq 6); do top -b -H -p <JIRA_PID> -n 1 > threaddump.$(date +%s).txt; kill -3 <JIRA_PID>; sleep 10; done

    For example, if the PID is 22668, enter:

    for i in $(seq 6); do top -b -H -p 22668 -n 1 > threaddump.$(date +%s).txt; kill -3 22668; sleep 10; done

    The output will be saved to threaddump.$(date +%s).txt in the current directory, where date +%s is the current Unix timestamp.

    You can modify the seq command parameter to adjust the the number of thread dumps to capture and the sleep command parameter to adjust the time between each thread dump.

    If you are connecting to the server through RDP, jstack might fail with following error:

    Not enough storage is available to process this command 

    You will need to open a RDP session in console mode: mstsc /admin

Linux/Unix Alternative: Generating thread dumps using scripts

You can also generate a thread dump by using scripts prepared by our Support team. That's an easy process, and the scripts will do everything for you. As an addition to generating a thread dump, the scripts also allow you to generate heap dumps, check the disk access speed, or the Java SSL connection.

  1. (Optional) Install the Thready app if it's supported by your Jira version. Thready renames Tomcat threads to include the URL they serve, which will make it easier to analyze the threads.
  2. Download and install the scripts from https://bitbucket.org/atlassianlabs/atlassian-support/.
  3. Execute the scripts when your Jira instance behaves slowly or is unresponsive. 
    1. (Optional) The scripts also allow you to test the disk access speed. This is described in more detail in Testing disk access speed. 
    2. When asked whether you want to capture thread dumps, enter Y.
    3. When asked whether you want to capture heap dumps, enter N.
    4. (Optional) The scripts also allow you to check the Java SSL connection.
  4. After running the scripts, the thread dump will be captured and compressed. You can now open a Support ticket and attach the generated package.

Analysis tools

Try TDA or Samurai to inspect your thread dump.

TDA

  1. Download TDA.
  2. CD to the directory where the JAR exists.

  3. Run:

    java -jar -Xmx512M ~/tda-bin-1.6/tda.jar
  4. Open your catalina.out file, containing the thread dump.

Issue processing thread dump with TDA (NumberFormatException)

Should you get an error on TDA console like:

Exception in thread "AWT-EventQueue-0" java.lang.NumberFormatException: For input string: "5 os_prio=0"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Long.parseLong(Long.java:441)
        at java.lang.Long.<init>(Long.java:702)
        at com.pironet.tda.utils.ThreadsTableModel.getValueAt(ThreadsTableModel.java:80)
        at com.pironet.tda.utils.TableSorter.getValueAt(TableSorter.java:285)
        at javax.swing.JTable.getValueAt(JTable.java:2717)
        at javax.swing.JTable.prepareRenderer(JTable.java:5719)
        at javax.swing.plaf.basic.BasicTableUI.paintCell(BasicTableUI.java:2114)
        at javax.swing.plaf.basic.BasicTableUI.paintCells(BasicTableUI.java:2016)
        at javax.swing.plaf.basic.BasicTableUI.paint(BasicTableUI.java:1812)
        at javax.swing.plaf.ComponentUI.update(ComponentUI.java:161)
        at javax.swing.JComponent.paintComponent(JComponent.java:778)
        at javax.swing.JComponent.paint(JComponent.java:1054)
        at javax.swing.JComponent.paintChildren(JComponent.java:887)
        at javax.swing.JComponent.paint(JComponent.java:1063)
        at javax.swing.JViewport.paint(JViewport.java:731)
        at javax.swing.JComponent.paintChildren(JComponent.java:887)
        at javax.swing.JComponent.paint(JComponent.java:1063)
        at javax.swing.JComponent.paintChildren(JComponent.java:887)
        at javax.swing.JSplitPane.paintChildren(JSplitPane.java:1047)
        at javax.swing.JComponent.paint(JComponent.java:1063)
        at javax.swing.JComponent.paintToOffscreen(JComponent.java:5230)
        at javax.swing.BufferStrategyPaintManager.paint(BufferStrategyPaintManager.java:295)
        at javax.swing.RepaintManager.paint(RepaintManager.java:1249)
        at javax.swing.JComponent._paintImmediately(JComponent.java:5178)
        at javax.swing.JComponent.paintImmediately(JComponent.java:4989)
        at javax.swing.RepaintManager$3.run(RepaintManager.java:808)
        at javax.swing.RepaintManager$3.run(RepaintManager.java:796)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
        at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:796)
        at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:769)
        at javax.swing.RepaintManager.prePaintDirtyRegions(RepaintManager.java:718)
        at javax.swing.RepaintManager.access$1100(RepaintManager.java:62)
        at javax.swing.RepaintManager$ProcessingRunnable.run(RepaintManager.java:1677)
        at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:312)
        at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:733)
        at java.awt.EventQueue.access$200(EventQueue.java:103)
        at java.awt.EventQueue$3.run(EventQueue.java:694)
        at java.awt.EventQueue$3.run(EventQueue.java:692)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:703)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:91)

Apply the following command on the thread dump(s) to fix the thread header format to make it processable:
sed -i 's/prio=[0-9]\{1,2\} os_prio=[0-9]\{1,2\}/prio=5/g' <filename>

For Mac:

sed -i '' 's/prio=[0-9]\{1,2\} os_prio=[0-9]\{1,2\}/prio=5/g' <filename>

Check the known thread dump knowledge base articles:

Last modified on Jul 12, 2021

Was this helpful?

Yes
No
Provide feedback about this article

Related content

Powered by Confluence and Scroll Viewport.