Problem

The confluence application occasionally crashes and throws an OutOfMemoryError as below in the catalina.out, stdout or atlassian-confluence.log:

Exception in thread "http-bio-8506-exec-106" java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:691)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:943)
	at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:992)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)

Diagnosis

Environment

• In certain cases on Solaris, it appears that this can also cause the Java application to completely crash and generate a core dump.

Cause

To provide concurrency (the ability to do multiple things at once), Java will spawn operating system threads and use them to perform tasks. Hard limits can be set at the system level on the number of threads that can be created, so if the application is requesting more threads than the OS is willing to provide, the above error is thrown. This occurs in the following way:

1. A new Java thread is requested by confluence applications. This can be by anything.
2. JVM native code proxies the request to create a new native thread to the OS.
3. The operating system attempts to create a new native thread. As it's a thread, it requires memory to be allocated to it.
4. The operating system refuses the native memory allocation.
5. The java.lang.OutOfMemoryError: unable to create new native thread error is thrown.

This can also happen if the operating system has no native memory left to allocate threads (say the 32-bit Java process space has been reached, or the OS virtual memory is fully depleted), or the maximum number of open files has been reached.

Workaround

1. Edit the $CONFLUENCE_INSTALL/bin/setenv.sh file and add the below to the top of the file:

   ulimit -u 4096
   ulimit -n 4096

    These values may be different depending upon the operating system used, consult with your System Administrator / hosting provider about the most appropriate limits to set.

2. Restart confluence.

It's recommended to set this value permanently as in the resolution.

Resolution

Setting the maximum running user processes and number of opened files permanently is recommended, and the implementation of this is operating system specific. These can be set in Debian / Ubuntu in limits.conf by adding the below:

confluence      soft   nofile  4096
confluence      hard   nofile  8192
confluence      soft   nproc   4096        
confluence      hard   nproc   8192

 Replace confluence with the user that runs Confluence.