Operations in Jira server fail with connection pool errors in logs

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform Notice: Server and Data Center Only - This article only applies to Atlassian products on the server and data center platforms.

Problem

Symptoms

  • The instance can present with symptoms of a performance problem - it appears requests are taking a long period of time to complete.
  • Requests fail in the browser.
  • JIRA functionality is failing.

The following appears in the atlassian-jira.log:

2015-03-04 16:27:41,193 ajp-bio-8009-exec-715 ERROR anonymous 976x372506x28 1i497bl 172.30.128.60 /secure/Dashboard.jspa [NoModule] There was an error getting a DBCP datasource.
java.lang.RuntimeException: Unable to obtain a connection from the underlying connection pool
	at org.ofbiz.core.entity.jdbc.interceptors.connection.ConnectionTracker.trackConnection(ConnectionTracker.java:59)
	at org.ofbiz.core.entity.transaction.DBCPConnectionFactory.trackConnection(DBCPConnectionFactory.java:242)
	at org.ofbiz.core.entity.transaction.DBCPConnectionFactory.getConnection(DBCPConnectionFactory.java:73)
	at org.ofbiz.core.entity.ConnectionFactory.tryGenericConnectionSources(ConnectionFactory.java:69)
	at org.ofbiz.core.entity.transaction.JNDIFactory.getConnection(JNDIFactory.java:146)
	at org.ofbiz.core.entity.TransactionFactory.getConnection(TransactionFactory.java:101)
	at org.ofbiz.core.entity.ConnectionFactory.getConnection(ConnectionFactory.java:59)
	at org.ofbiz.core.entity.jdbc.SQLProcessor.getConnection(SQLProcessor.java:367)
	at org.ofbiz.core.entity.GenericDAO.select(GenericDAO.java:584)
	at org.ofbiz.core.entity.GenericHelperDAO.findByPrimaryKey(GenericHelperDAO.java:97)
	at org.ofbiz.core.entity.GenericDelegator.findByPrimaryKey(GenericDelegator.java:609)
	at org.ofbiz.core.entity.GenericDelegator.findByPrimaryKey(GenericDelegator.java:647)
	...
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object
	at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:114)
	at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
	at org.ofbiz.core.entity.transaction.DBCPConnectionFactory$1.call(DBCPConnectionFactory.java:246)
	at org.ofbiz.core.entity.transaction.DBCPConnectionFactory$1.call(DBCPConnectionFactory.java:243)
	at org.ofbiz.core.entity.jdbc.interceptors.connection.ConnectionTracker.trackConnection(ConnectionTracker.java:53)
	... 236 more
Caused by: java.util.NoSuchElementException: Timeout waiting for idle object
	at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1134)
	at org.apache.commons.dbcp.AbandonedObjectPool.borrowObject(AbandonedObjectPool.java:79)
	at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
	... 240 more

Or the below may also appear:

2016-01-12 11:24:05,094 StreamsCompletionService::thread-28582 ERROR potatobake 683x843721x16 1mrieu1 127.0.0.1 /plugins/servlet/streams [NoModule] There was an error getting a DBCP datasource.
java.lang.RuntimeException: Unable to obtain a connection from the underlying connection pool
	at org.ofbiz.core.entity.jdbc.interceptors.connection.ConnectionTracker.trackConnection(ConnectionTracker.java:60)
	at org.ofbiz.core.entity.transaction.DBCPConnectionFactory.trackConnection(DBCPConnectionFactory.java:243)
	at org.ofbiz.core.entity.transaction.DBCPConnectionFactory.getConnection(DBCPConnectionFactory.java:74)
	at org.ofbiz.core.entity.ConnectionFactory.tryGenericConnectionSources(ConnectionFactory.java:69)
	at org.ofbiz.core.entity.transaction.JNDIFactory.getConnection(JNDIFactory.java:146)
	at org.ofbiz.core.entity.TransactionFactory.getConnection(TransactionFactory.java:136)
	at org.ofbiz.core.entity.ConnectionFactory.getConnection(ConnectionFactory.java:59)
	at com.atlassian.jira.ofbiz.DefaultOfBizConnectionFactory.getConnection(DefaultOfBizConnectionFactory.java:52)
	at com.atlassian.activeobjects.jira.JiraTenantAwareDataSourceProvider$OfBizDataSource.getConnection(JiraTenantAwareDataSourceProvider.java:125)
	at com.atlassian.activeobjects.internal.DataSourceProviderActiveObjectsFactory$ActiveObjectsDataSource.getConnection(DataSourceProviderActiveObjectsFactory.java:93)
	...
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, general error
	at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:118)
	at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
	at org.ofbiz.core.entity.transaction.DBCPConnectionFactory$1.call(DBCPConnectionFactory.java:247)
	at org.ofbiz.core.entity.transaction.DBCPConnectionFactory$1.call(DBCPConnectionFactory.java:244)
	at org.ofbiz.core.entity.jdbc.interceptors.connection.ConnectionTracker.trackConnection(ConnectionTracker.java:54)
Caused by: java.lang.InterruptedException
	at java.lang.Object.wait(Native Method)
	at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1112)
	at org.apache.commons.dbcp.AbandonedObjectPool.borrowObject(AbandonedObjectPool.java:79)
	at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
	... 187 more

Cause

JIRA uses a database connection pool to manage connections with the database. When an action within JIRA requests data from the database, it will go through a persistence layer such as Active Objects or OfBiz which will then query the database. When it queries the database, it uses up a connection, which is retrieved from the connection pool.

This timeout happens when the connection pool has been exhausted (as all the connections are currently in use) and can be caused by any of the following:

Workaround

  1. Ensure the recommended dbconfig.xml is being used from the appropriate database docs as specified in Connecting JIRA to a Database. Please note this is version-specific functionality so ensure the appropriate documentation is being used.
  2. Increasing the number of connections can mitigate this problem, for example increasing pool-max-size in the dbconfig.xml, say changing it from 20 to 50. This will not have any side effects as far as increased overhead on resources.

    Please ensure the database has enough active connections to serve the maximum set in JIRA. To verify this consult with your DBA or the database documentation.

  3. To identify leaking connections, logAbandoned can be added to dbconfig.xml inside the jdbc-datasource tag, for example in PostgreSQL:

    <jira-database-config>
      <name>defaultDS</name>
      <delegator-name>default</delegator-name>
      <database-type>postgres72</database-type>
      <schema-name>public</schema-name>
      <jdbc-datasource>
        <url>jdbc:postgresql://dbserver:5432/jiradb</url>
        <driver-class>org.postgresql.Driver</driver-class>
        <username>jiradbuser</username>
        <password>password</password>
        <pool-min-size>20</pool-min-size>
        <pool-max-size>20</pool-max-size>
        <pool-max-wait>30000</pool-max-wait>
        <pool-max-idle>20</pool-max-idle>
        <pool-remove-abandoned>true</pool-remove-abandoned>
        <pool-remove-abandoned-timeout>300</pool-remove-abandoned-timeout>
     
        <validation-query>select version();</validation-query>
        <min-evictable-idle-time-millis>60000</min-evictable-idle-time-millis>
        <time-between-eviction-runs-millis>300000</time-between-eviction-runs-millis>
       
        <pool-test-on-borrow>false</pool-test-on-borrow>
        <pool-test-while-idle>true</pool-test-while-idle>
        <logAbandoned>true</logAbandoned>
      </jdbc-datasource>
    </jira-database-config>

    This will add additional overhead as stack traces will be thrown in the logs for abandoned connections, which can have a performance impact.

  4. Identifying SQL that is taking longer to process than normal can be done by Enable SQL query logging in Jira server.

Jira slow with dangerous use of multiple connections error in log is also another known problem that can occur as a result of an improperly configured dbconfig.xml.

Description

This page covers how to resolve the issue of Jira Stopping Functions due to Timeout waiting for idle object.

ProductJira
Last modified on Nov 17, 2020

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.