Summary

The Cron-based scheduling in a Build plan is not working as expected. We could see that the scheduler was triggering, but the Build did not run.

Environment

All supported versions of Bamboo

Diagnosis

You could see the below logs in the <bamboo-home>/logs/atlassian-bamboo.log as it was triggered, but no signs of the Build plan executed.

2023-02-19 22:45:00,000 INFO [scheduler_Worker-9] [QuartzTriggerJob] Running quartz strategy job for CHAIN:BRO-VAI
2023-02-19 22:45:00,001 INFO [scheduler_Worker-2] [QuartzTriggerJob] Running quartz strategy job for CHAIN:TELL-CAI
2023-02-19 22:45:00,001 INFO [scheduler_Worker-1] [QuartzTriggerJob] Running quartz strategy job for CHAIN:XDE-VOP
2023-02-19 22:45:00,001 INFO [scheduler_Worker-5] [QuartzTriggerJob] Running quartz strategy job for CHAIN:COL-CAR
2023-02-19 22:45:00,001 INFO [scheduler_Worker-4] [QuartzTriggerJob] Running quartz strategy job for CHAIN:JIR-LET

Also, you can see <Bamboo_Base_URL>/admin/viewBambooInternals.action page 'Plan Execution Stats' section that these threads are busy and you have a huge list of 'Pending events.'

Cause

Bamboo has 4 PlanExec threads responsible for moving builds through the early execution phases. These threads can get tied up servicing change detection operations if VCS operations are slow (retired repository references in the plans) or your Bamboo instance has a high count of repository polling occurring at high frequency. 

Solution

Solution 1

Upgrade to Bamboo 9.1.0  above as the bug BAM-20259-Bamboo PlanExec default size could benefit from an increase fixed in this version to increase the PlanExec threads dynamically based on load factors and queue drain rate.

Solution 2

Increase the amount of plan execution threads by adding the following start-up argument if you have >= 8 CPU cores on your Bamboo server.       

1. Linux: Edit your <bamboo-install>/bin/setenv.sh
   • Windows, please see the following on how to define additional arguments depending on the installation: https://confluence.atlassian.com/bamboo/configuring-your-system-properties-289277345.html

2. Add the following -Dbamboo.plan.exe.threads=8 to your JVM_SUPPORT_RECOMMENDED_ARGS. E.g

   JVM_SUPPORT_RECOMMENDED

   #
   #  Occasionally Atlassian Support may recommend that you set some specific JVM arguments.  You can use this variable below to do that.
   #
   : ${JVM_SUPPORT_RECOMMENDED_ARGS:="-Dbamboo.plan.exe.threads=8"}

3. Restart Bamboo

If none of the above solutions works, please generate a Bamboo Support Zip with  a thread dump in the Bamboo application server  and contact Atlassian Support to investigate further.