JIRA threads grow uncontrollably with lots of Analytics threads in Timed_Waiting state

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Problem

JIRA threads grow uncontrollably (a few thousands) exceeding the maxThreads specified in the Connector used by JIRA in server.xml. Due to this, JIRA may run out of resources and throw this error:

21-Feb-2017 13:51:46.985 SEVERE [ajp-nio-8009-exec-148] org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at com.sun.jndi.ldap.Connection.<init>(Connection.java:234)
at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:137)

Diagnosis

A lot of these threads are found in thread dumps: Analytics in Timed_Waiting state

"Analytics" prio=1 tid=0x0000000000004645 nid=0 sleeping 
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at com.segment.backo.Backo.sleep(Backo.java:72)
	at com.segment.analytics.internal.AnalyticsClient$BatchUploadTask.run(AnalyticsClient.java:158)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at com.segment.analytics.Platform$1$1.run(Platform.java:44)
	at java.lang.Thread.run(Thread.java:745)
   Locked ownable synchronizers:
	- java.util.concurrent.ThreadPoolExecutor$Worker@@8a291d

Cause

The thread above comes from Tempo Analytics, which is bundled in one of the Tempo plugins. The keyword that helps identify this, in this case, is com.segment.

The way to find which plugin is causing this is to unpack the installed plugins' .jar files and then grep for the class name in all the unpacked folders. E.g. for this case:

grep -r "com.segment" ./
Be aware that ScriptRunner plugin also uses *com.segment.analytics* classes for analytics purposes.

Workaround

Go to JIRA Administration -> Add-ons -> Tempo Analytics -> disable Analytics. This will prevent the Analytics threads from accumulating.

Resolution

tip/resting Created with Sketch.

According to Tempo for JIRA, the issue is fixed in Tempo Timesheets 8.1.3. Release notes can be found here.

Last modified on Dec 20, 2019

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.