Remote agents are disconnected due to the ActiveMQ broker blocking all queues
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Prior to Bamboo 9.4, a remote agent’s default socket inactivity timeout period was 5 minutes. This meant that if a remote agent terminated abruptly, Bamboo wouldn’t close the transport to the terminated agent for up to 5 minutes. If the dead transport was used, ActiveMQ could potentially close all queues, disconnecting all remote agents.
In Bamboo 9.4, the issue was addressed for fresh instances by reducing the maximum inactivity duration to 90 seconds and introducing a socket timeout of 45 seconds on the transport. The configuration of active instances upgraded to Bamboo 9.4 must be updated manually.
Diagnosis
All remote agents disconnect abruptly.
Cause
The default socket inactivity timeout causes dead ActiveMQ transports to be kept open for up to 5 minutes after the abrupt termination of a remote agent.
Solution
If you’ve just installed and configured a fresh instance of Bamboo 9.4, no manual intervention is required. In Bamboo 9.4, the new inactivity duration and socket timeout values are applied to the broker and broker client by default.
If you’ve just upgraded an active instance to Bamboo 9.4, manually apply the inactivity duration and socket timeout values to the broker and broker client URLs. You can do that in the Bamboo web interface or by editing the bamboo.cfg.xml configuration file.
To apply the changes to the broker and broker client URLs through the Bamboo web interface:
From the upper-right corner of the screen, select Administration
> Overview.
In the menu on the left, under System, select General configuration.
In the Broker URL field, set the
wireFormat.maxInactivityDurationparameter to90000and add thesoWriteTimeoutparameter with a value of45000. For example:nio://0.0.0.0:54663?wireFormat.maxInactivityDuration=90000&transport.soWriteTimeout=45000In the Broker client URL field, set the
wireFormat.maxInactivityDurationparameter to90000. For example:failover:(nio://0.0.0.0:54663?wireFormat.maxInactivityDuration=90000)?initialReconnectDelay=15000&maxReconnectAttempts=10The broker client URL doesn’t expect the
transport.soWriteTimeoutproperty.- Select Save.
- Restart the Bamboo server.