Problem

If you have Bamboo running against an Aurora database cluster in the Amazon Relational Database Service (RDS), with:

• The default Bamboo Hibernate configuration in <bamboo-home>/bamboo.cfg.xml,
• Database instance size db.t2.medium
  
  

The following appears in <bamboo-home>/logs/atlassian-bamboo.log:

2017-09-15 14:46:35,150 WARN [C3P0PooledConnectionPoolManager[identityToken->2rvy6z9qzw98y31mxen6c|6c4bdbc3]-HelperThread-#2] [BasicResourcePool] com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@768e3d7a -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (30). Last acquisition attempt exception: 
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
...
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Too many connections
...

Cause

The maximum number of connections allowed to an Aurora DB instance is determined by the max_connections parameter in the instance-level parameter group for the DB instance.

The Managing an Amazon Aurora DB Cluster documentation from Amazon Web Services (AWS) shows the default value of max_connections in the table below:

AWS calculates this value using the following formula, where log represents log2 (log base 2), and :

GREATEST({log(DBInstanceClassMemory/805306368)*45},{log(DBInstanceClassMemory/8187281408)*1000})

Workaround

To work around this issue, you can update the hibernate.c3p0.max_size variable in bamboo.cfg.xml to be 90 or less. It is preferable to go under the limit, so that connections from Bamboo don't clash with any others from other applications or AWS monitoring.

An estimated database connection pool size for your Bamboo instance can be calculated with the following formula:

(Concurrent users)/5 + (Busy remote agents)/5 + (Local agents)*1.1 + (Amount of concurrent change detections)

Please see Bamboo Best Practice - System Requirements - Database connection pool size for more information and an worked example of this formula. 

Resolution

Per AWS, you can increase the maximum number of connections to your Aurora DB instance by scaling the instance up to a DB instance class with more memory, or by setting a larger value for the max_connections parameter, up to 16,000.