Bamboo Server fails to Initialize Plans on Server Restart

Still need help?

The Atlassian Community is here for you.

Ask the community

Symptoms

Bamboo Server fails to initialize plans and fails with the following exception

 

2011-10-12 11:16:16,966 FATAL [main] [BambooContainer] Cannot start bamboo
java.lang.ArrayIndexOutOfBoundsException: -1
at java.util.ArrayList.set(ArrayList.java:339)
at net.sf.hibernate.collection.List.readFrom(List.java:312)
at net.sf.hibernate.loader.Loader.readCollectionElement(Loader.java:384)
at net.sf.hibernate.loader.Loader.getRowFromResultSet(Loader.java:240)
......
......
at $Proxy11.getAllPlans(Unknown Source)
at com.atlassian.bamboo.container.BambooContainer.initialisePlans(BambooContainer.java:332)
at com.atlassian.bamboo.container.BambooContainer.start(BambooContainer.java:243) 

OR

2013-01-04 14:28:04,855 FATAL [main] [BambooContainer] Cannot start bamboo
com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException
	at com.google.common.cache.CustomConcurrentHashMap$ComputedUncheckedException.get(CustomConcurrentHashMap.java:3305)
	at com.google.common.cache.CustomConcurrentHashMap$ComputingValueReference.compute(CustomConcurrentHashMap.java:3441)
	at com.google.common.cache.CustomConcurrentHashMap$Segment.compute(CustomConcurrentHashMap.java:2322)
	at com.google.common.cache.CustomConcurrentHashMap$Segment.getOrCompute(CustomConcurrentHashMap.java:2291)
	at com.google.common.cache.CustomConcurrentHashMap.getOrCompute(CustomConcurrentHashMap.java:3802)
......
Caused by: java.lang.NullPointerException
	at com.atlassian.bamboo.deletion.NotDeletedPredicate.apply(NotDeletedPredicate.java:27)
	at com.atlassian.bamboo.deletion.NotDeletedPredicate.apply(NotDeletedPredicate.java:8)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)

Cause

This is caused due to corruption in the Bamboo database, the LIST_POSITION column of CHAIN_STAGE table which stores the list order of Bamboo jobs is corrupted and has a value of -1. Another possible corruption of LIST_POSITION column is when there are duplicate LIST_POSITION values for the same BUILD_ID value in CHAIN_STAGE table.

Resolution

1. To check if you're affected by this bug execute the following command:

select BUILD_ID from CHAIN_STAGE where LIST_POSITION= -1;

If the output returns any results, execute the following SQL;

select * from CHAIN_STAGE where BUILD_ID=<VALUE>; 
 

 STAGE_ID CREATED_D UPDATED_D NAME DESCRIPTION BUILD_ID LIST_POSITION MARKED_FOR_DELETION OPTIONAL_STAGE
---------- --------- --------- -------------------- ------------------------------------------------------- ---------- ------------- ------------------- --------------
 71401503 Smoke Test JOB_2 58097680 -1 0 0
 61538457 Build JOB_0 58097680 0 0 0
 71401504 Release JOB_1 58097680 1 0 0


In the example above JOB_2 ha sa LIST_POSITION of -1, instead of 2.

 

2. Another example of this corruption is when there are duplicate LIST_POSITIONS for the same BUILD_ID. For instance, this listing looks fine:

because for the same BUILD_ID there are no duplicate LIST_POSITION values. If there were LIST_POSITION 3 and another 3 instead of 2 and 3 for the same BUILD_ID 7176214, then that would be a corruption. The easiest way to find out this type of corruptions is to enable detailed SQL logging, reproduce the problem, and see where exactly Bamboo fails. Based on the queries that Bamboo tries to run right before failing, we can identify the problematic plan. Next, we can check the related details in the CHAIN_STAGE table.

 

 

3. The final step is to find the problematic List Position for your build and execute the SQL below:

update CHAIN_STAGE set LIST_POSITION = <VALUE> where STAGE_ID = <VALUE>;
Last modified on Nov 15, 2018

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.