Summary

The following message appears frequently in the atlassian-crowd.log.

C3P0PooledConnectionPoolManager[identityToken->2so7snad10br9smlhh0jq|7dd5f43f]-AdminTaskTimer WARN [mchange.v2.async.ThreadPoolAsynchronousRunner] com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@4e9df9d2 -- APPARENT DEADLOCK!!! Complete Status: 
Managed Threads: 3
Active Threads: 3
Active Tasks: 
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@1203adc7
on thread: C3P0PooledConnectionPoolManager[identityToken->2so7snad10br9smlhh0jq|7dd5f43f]-HelperThread-#2
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@1e5bb8f7
on thread: C3P0PooledConnectionPoolManager[identityToken->2so7snad10br9smlhh0jq|7dd5f43f]-HelperThread-#1
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask@4dc74be2
on thread: C3P0PooledConnectionPoolManager[identityToken->2so7snad10br9smlhh0jq|7dd5f43f]-HelperThread-#0

This may cause performance issues with Crowd or users may not be able to authenticate to connected applications during these times.

Environment

• Crowd Server or Data Center

• Any connected database

Cause

This error can indicate issues with the database, its connection pool, and/or the database's ability to cleanly acquire a new connection.

Resolution

Restart of Crowd is required after making any of the below changes

• Optimize the database connection pool. This defines the maximum number of connections in the database pool. This number should be at least 10 counts higher or 25% higher (whichever value is greater) than the number of HTTP threads (defined under <Crowd_Install_Directory>/apache-tomcat/conf/server.xml file using parameter maxThreads). This is to account for connections used for background jobs on top of active HTTP threads.
  To set the database connection pool:

  • If you are using a direct JDBC connection to the database:
    Navigate to the <Crowd_Home_Directory>/crowd.cfg.xml and you will find the following property which should be updated as per your need:

    <property name="hibernate.c3p0.max_size">30</property>
    

  • If you are using a data source to connect to the database in <Crowd_Install_Directory>/apache-tomcat/conf/server.xml, configure the maxTotal parameter

• Also confirm if a validation query exists to check for broken connections and clean them from the availability pool. This is under:
  • <Crowd_Home_Directory>/crowd.cfg.xml file, and check if the following property exists, and if not add them manually:

    • For PostgreSQL, MSSQL, MySQL:

      <property name="hibernate.c3p0.preferredTestQuery">select 1</property>
      

    • For Oracle:

      <property name="hibernate.c3p0.preferredTestQuery">select 1 from dual</property>