Jira server throws OutOfMemoryError: unable to create new native thread
The JIRA application occasionally crashes and throws an
OutOfMemoryError as below in the
Exception in thread "http-bio-8506-exec-106" java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:691) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:943) at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:992) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722)
The above error message will be present in the logs. This can be different error messages associated with an
OutOfMemoryError, this one can be identified by the "unable to create new native thread" error message. We have different KBs for others as below:
- Jira server crashes with OutofMemory Java heap space error
- GC overhead limit exceeded error crashes Jira server
- JIRA applications crash due to OutOfMemoryError PermGen space error
- OutOfMemoryError: Metaspace errors with Java 8+ in Jira server
To provide concurrency (the ability to do multiple things at once), Java will spawn operating system threads and use them to perform tasks. There can be hard-limits on the number of threads created by the operating system, so if the application is requesting more threads that the OS is willing to provide, the above error will be thrown. This occurs in the following way:
- A new Java thread is requested by JIRA applications. This can be by anything.
- JVM native code proxies the request to create a new native thread to the OS.
- The operating system attempts to create a new native thread. As it's a thread, it requires memory to be allocated to it.
- The operating system refuses the native memory allocation.
java.lang.OutOfMemoryError: unable to create new native threaderror is thrown.
This can also happen if the operating system has no native memory left to allocate threads (say the 32-bit Java process space has been reached, or the OS virtual memory is fully depleted), or the maximum number of open files has been reached.
In certain cases on Solaris it appears that this can also cause the Java application to completely crash and generate a core dump.
$JIRA_INSTALL/bin/setenv.shfile and add the below to the top of the file:
ulimit -u 16384 ulimit -n 16384
These values may be different depending upon the operating system used, consult with your System Administrator / hosting provider about the most appropriate limits to set.
- The changes can be verified by running
/proc/<pid>/limitswhere <pid> is the application process ID.
It's recommended to set this value permanently as in the resolution.
Setting the maximum running user processes and number of opened files permanently is recommended, and the implementation of this is operating system-specific. These can be set in Debian / Ubuntu in limits.conf by adding the below:
jira soft nofile 16384 jira hard nofile 32768 jira soft nproc 16384 jira hard nproc 32768
jira with the user that runs JIRA.