JIRA threads grow uncontrollably with lots of Analytics threads in Timed_Waiting state

Still need help?

The Atlassian Community is here for you.

Ask the community

This article only applies to Atlassian's server products. Learn more about the differences between cloud and server.

Problem

JIRA threads grow uncontrollably (a few thousands) exceeding the maxThreads specified in the Connector used by JIRA in server.xml. Due to this, JIRA may run out of resources and throw this error:

21-Feb-2017 13:51:46.985 SEVERE [ajp-nio-8009-exec-148] org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at com.sun.jndi.ldap.Connection.<init>(Connection.java:234)
at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:137)

Diagnosis

A lot of these threads are found in thread dumps: Analytics in Timed_Waiting state

"Analytics" prio=1 tid=0x0000000000004645 nid=0 sleeping 
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at com.segment.backo.Backo.sleep(Backo.java:72)
	at com.segment.analytics.internal.AnalyticsClient$BatchUploadTask.run(AnalyticsClient.java:158)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at com.segment.analytics.Platform$1$1.run(Platform.java:44)
	at java.lang.Thread.run(Thread.java:745)
   Locked ownable synchronizers:
	- java.util.concurrent.ThreadPoolExecutor$Worker@@8a291d

Cause

The thread above comes from Tempo Analytics, which is bundled in one of the Tempo plugins. The keyword that helps identify this, in this case, is com.segment.

The way to find which plugin is causing this is to unpack the installed plugins' .jar files and then grep for the class name in all the unpacked folders. E.g. for this case:

grep -r "com.segment" ./

Workaround

Go to JIRA Administration -> Add-ons -> Tempo Analytics -> disable Analytics. This will prevent the Analytics threads from accumulating.

Resolution

tip/resting Created with Sketch.

According to Tempo for JIRA, the issue is fixed in Tempo Timesheets 8.1.3. Release notes can be found here.

Last modified on Nov 6, 2018

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.