Agent remains in loading: "java.net.BindException: Address already in use"
Problem
Agents are not connecting and but the server is able to see them loading.
From Cog icon >> Overview >> Agents >> Online remote agents (tab) the following message can be seen:
Nov 12, 2017 2:55:32 PM A remote agent is loading on 192.168.1.203 (127.0.0.1).Diagnosis
One of the possible causes could be the address already in use. If this is the case the following can be found in the server logs:
2017-11-12 14:48:44,686 ERROR [localhost-startStop-1] [BambooBrokerService] Unable to start JMS connector at nio://0.0.0.0:54663?wireFormat.maxInactivityDuration=300000
java.io.IOException: Failed to bind to server socket: nio://0.0.0.0:54663?wireFormat.maxInactivityDuration=300000 due to: java.net.BindException: Address already in use
at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:34)
at org.apache.activemq.transport.tcp.TcpTransportServer.bind(TcpTransportServer.java:143)
at org.apache.activemq.transport.tcp.TcpTransportFactory.doBind(TcpTransportFactory.java:62)
at org.apache.activemq.transport.TransportFactorySupport.bind(TransportFactorySupport.java:40)
at org.apache.activemq.broker.BrokerService.createTransportConnector(BrokerService.java:2478)
at org.apache.activemq.broker.BrokerService.addConnector(BrokerService.java:338)
at com.atlassian.bamboo.amq.BambooBrokerService.startConnector(BambooBrokerService.java:128)
at com.atlassian.bamboo.amq.BambooBrokerService.addStartedConnectors(BambooBrokerService.java:108)
at com.atlassian.bamboo.buildqueue.manager.RemoteAgentManagerImpl.startOrStopConnectors(RemoteAgentManagerImpl.java:422)
at com.atlassian.bamboo.buildqueue.manager.RemoteAgentManagerImpl.start(RemoteAgentManagerImpl.java:539)
at com.atlassian.bamboo.container.BambooContainer.start(BambooContainer.java:265)
at com.atlassian.bamboo.upgrade.UpgradeLauncher.upgradeAndStartBamboo(UpgradeLauncher.java:131)
at com.atlassian.bamboo.upgrade.UpgradeLauncher.contextInitialized(UpgradeLauncher.java:42)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4842)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5303)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1407)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1397)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.apache.activemq.transport.nio.NIOTransportFactory$2.createServerSocket(NIOTransportFactory.java:83)
at org.apache.activemq.transport.tcp.TcpTransportServer.bind(TcpTransportServer.java:140)
... 20 moreCause
This mostly happens when two instances of Bamboo are running on the same server. It could be the case of having both production and development instances with broker configured to listen on 54663.
In cases where there is only one Bamboo instance and binding error still remains, the netstat command can be helpful.
The following can be used to diagnose what process is holding the port:
| OS | Command |
|---|---|
| Windows | netstat-an | findstr54663 |
| Unix like | netstat -an | grep 54663 |
Resolution
Shutdown the instance with the error Address already in use
Edit the
<Bamboo_Home>/bamboo.cfg.xmlfile and update the port in the following two lines:<property name="bamboo.jms.broker.client.uri">failover:(tcp://bamboo_server_host:54663?wireFormat.maxInactivityDuration=300000)?maxReconnectAttempts=10&initialReconnectDelay=15000</property> <property name="bamboo.jms.broker.uri">tcp://0.0.0.0:54663?wireFormat.maxInactivityDuration=300000</property>This change can be performed also from the Bamboo UI (Cog icon >> Overview >> General configuration) in versions above 5.x.
Restart your Bamboo server for the changes to take effect.