Summary

In some edge cases, a script can be used to take action on the remote agent, such as shutting down and restarting the remote agent based on a certain trigger (Disk full, Memory usage exceeding a certain threshold etc.).

By default, when a remote agent shutdown is triggered by Atlassian shipped agent management script, the remote agent will complete its currently running Job before shutting down, however, if the agent process is forcefully killed by a  custom script, the agent's currently running jobs will be abnormally stopped which can lead to an unpredictable build error, and a similar error can be seen in the agent log.

2022-12-13 04:50:43,854 INFO [scheduler_Worker-8] [OrphanedBuildMonitorJob] PLAN-TEST160-JOB1-1 marked as InProgress but not present in CBC since Tue Dec 13 04:37:43 CET 2023
2022-12-13 04:50:43,854 ERROR [scheduler_Worker-8] [OrphanedBuildMonitorJob] Build PLAN-TEST160-JOB1-1 had to be cancelled: it was marked as in progress in DB but no agents were assigned to it.

Environment

• Remote agents on both DC and standalone instance.

Solution

• Optimise the cleanup script by first stopping the agent process with the action request for agent stop gracefully.

  • With this API call, If the remote agent is building a job, it will be gracefully shut down after it has finished building the current job, else it will shut down the remote agent after 200ms if the agent is not building.

    curl -u uname"pass -X POST -H "X-Atlassian-Token: no-check" --url "http://localhost:6990/bamboo/admin/agent/stopAgentNicely.action?agentId=1081345"

     Note that because this is not an API endpoint, the personal access token will not work for it, so HTTP basic authentication username and password need to be used.