Poor performance and a frozen instance due to a bug in Java
Symptoms
Following shows up in Jstack thread dumps:
Found one Java-level deadlock:
=============================
"TestServer-exec-240":
waiting to lock monitor 0x000000005f224878 (object 0x000000050776de70, a java.util.WeakHashMap),
which is held by "Thread-51"
"Thread-51":
waiting to lock monitor 0x000000005f6a9660 (object 0x000000050ead0068, a com.sun.jndi.ldap.Connection),
which is held by "Thread-2100"
"Thread-2100":
waiting to lock monitor 0x000000005dcd6268 (object 0x000000050d093e18, a com.sun.jndi.ldap.pool.Connections),
which is held by "Thread-51"
Java stack information for the threads listed above:
===================================================
"TestServer-exec-240":
at com.sun.jndi.ldap.pool.Pool.getPooledConnection(Pool.java:109)
- waiting to lock <0x000000050776de70> (a java.util.WeakHashMap)
at com.sun.jndi.ldap.LdapPoolManager.getLdapClient(LdapPoolManager.java:310)
at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1572)
at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2678)
......
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:200)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
- locked <0x00000005a08c5010> (a org.apache.tomcat.util.net.SocketWrapper)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
"Thread-51":
at com.sun.jndi.ldap.Connection.cleanup(Connection.java:617)
- waiting to lock <0x000000050ead0068> (a com.sun.jndi.ldap.Connection)
at com.sun.jndi.ldap.LdapClient.forceClose(LdapClient.java:441)
at com.sun.jndi.ldap.LdapClient.closeConnection(LdapClient.java:459)
- locked <0x000000050ead0320> (a com.sun.jndi.ldap.LdapClient)
at com.sun.jndi.ldap.pool.ConnectionDesc.expire(ConnectionDesc.java:113)
- locked <0x000000050ead4748> (a com.sun.jndi.ldap.pool.ConnectionDesc)
at com.sun.jndi.ldap.pool.Connections.expire(Connections.java:280)
- locked <0x000000050d093e18> (a com.sun.jndi.ldap.pool.Connections)
at com.sun.jndi.ldap.pool.Pool.expire(Pool.java:153)
- locked <0x000000050776de70> (a java.util.WeakHashMap)
at com.sun.jndi.ldap.pool.PoolCleaner.run(PoolCleaner.java:44)
- locked <0x000000050776db68> (a com.sun.jndi.ldap.pool.PoolCleaner)
"Thread-2100":
at com.sun.jndi.ldap.pool.Connections.removePooledConnection(Connections.java:247)
- waiting to lock <0x000000050d093e18> (a com.sun.jndi.ldap.pool.Connections)
at com.sun.jndi.ldap.LdapClient.processConnectionClosure(LdapClient.java:482)
at com.sun.jndi.ldap.Connection.cleanup(Connection.java:659)
- locked <0x000000050ead0068> (a com.sun.jndi.ldap.Connection)
at com.sun.jndi.ldap.Connection.run(Connection.java:959)
at java.lang.Thread.run(Thread.java:662)
Found 1 deadlock.
------------------------------------------------
Cause
The problem is caused by a bug in JAVA: Bug # 7174887: Deadlock in jndi ldap connection cleanup which is fixed in Java™ SE Development Kit 7 Update 40 Bug Fixes.
Resolution
Update Java to 1.7.0.40 or a newer version that is compatible.
Make sure to modify your variables accordingly after this change and re-start. (ex.: BAMBOO_HOME).
Last modified on Dec 5, 2019
Powered by Confluence and Scroll Viewport.