Remote agents are unable to validate the failover and connect to the server
Symptoms
Remote agent fails to connect to the server and the following errors show up in the logs:
2014-01-28 07:45:03,165 WARN [Thread-1] [BambooActiveMQConnectionFactory] Unable to validate failover:(tcp://10.30.5.140:54671?wireFormat.maxInactivityDuration=300000)?initialReconnectDelay=15000&maxReconnectAttempts=10: java.net.URISyntaxException: Illegal character in scheme name at index 0: 10.33.2.140:54671?wireFormat.maxInactivityDuration=300000
2014-01-28 07:49:18,822 ERROR [ActiveMQ Task-1] [FailoverTransport] Failed to connect to [tcp://10.33.2.140:54671?wireFormat.maxInactivityDuration=300000] after: 10 attempt(s)
2014-01-28 07:49:18,824 WARN [Thread-1] [BambooJmsProxyFactoryBean] Exception when running afterPropertiesSet
javax.jms.JMSException: Connection refused
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1380)
at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1466)
at org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:503)
at org.apache.activemq.pool.ConnectionPool.start(ConnectionPool.java:90)
at org.apache.activemq.pool.PooledConnection.start(PooledConnection.java:87)
at org.logicblaze.lingo.jms.JmsProducerConfig.createConnection(JmsProducerConfig.java:87)
....INFO | jvm 10 | 2014/01/30 22:33:10 | 2014-01-30 22:32:56,575 WARN [Thread-1] [BambooActiveMQConnectionFactory] Unable to validate failover:(tcp://0.0.0.0:54671?wireFormat.maxInactivityDuration=300000)?initialReconnectDelay=15000&maxReconnectAttempts=10: java.net.URISyntaxException: Illegal character in scheme name at index 0: 0.0.0.0:54671?wireFormat.maxInactivityDuration=300000
INFO | jvm 10 | 2014/01/30 22:33:10 | 2014-01-30 22:32:56,748 INFO [Thread-1] [BambooActiveMQConnectionFactory] Setting broker URL to 'failover:(tcp://0.0.0.0:54671?wireFormat.maxInactivityDuration=300000)?initialReconnectDelay=15000&maxReconnectAttempts=10'
INFO | jvm 10 | 2014/01/30 22:33:10 | 2014-01-30 22:32:56,754 WARN [Thread-1] [BambooActiveMQConnectionFactory] Unable to validate failover:(tcp://0.0.0.0:54671?wireFormat.maxInactivityDuration=300000)?initialReconnectDelay=15000&maxReconnectAttempts=10: java.net.URISyntaxException: Illegal character in scheme name at index 0: 0.0.0.0:54671?wireFormat.maxInactivityDuration=300000
STATUS | wrapper | 2014/01/30 22:36:15 | INT trapped. Shutting down.Scenario
A new Bamboo instance is installed on the same machine as an existing instance. The existing instance is using TCP port 54663 for the broker port number.
The new Bamboo instance is configured to use a different port, in the example above 54671, for the broker where the remote agents connect.
The old instance should stay alive until the new instance works correctly and has all the remote agents connected to it and can run builds successfully.
Cause
The remote agent cannot connect to the TCP port 54671 as configured for the broker port on the Bamboo server side.
Resolution
Though it's not recommended to use a different port, if it's necessary to switch such as when running multiple Bamboo server instances on the same machine, only ONE port should be used for both "bamboo.jms.broker.client.uri" and "bamboo.jms.broker.uri". For example, for the new instance that would use TCP port 54671 for the broker:
<property name="bamboo.jms.broker.client.uri">failover:(tcp://abcd.testcompany.com/bamboo:54671?wireFormat.maxInactivityDuration=300000)?initialReconnectDelay=15000&maxReconnectAttempts=10</property>
<property name="bamboo.jms.broker.uri">tcp://0.0.0.0:54671?wireFormat.maxInactivityDuration=300000</property>If modifying "bamboo.jms.broker.client.uri" and "bamboo.jms.broker.uri" from the UI (Bamboo Administration - > General Configuration), please note that they are in a different order than if you are editing them in the "bamboo.cfg.xml" file under <bamboo-home> directory:
Perform the following steps:
- Verify that one port used for "bamboo.jms.broker.client.uri" and "bamboo.jms.broker.uri" and the configuration in the UI or <bamboo-home>/bamboo.cfg.xml file is the expected one.
- Verify that the configured TCP port for the broker is open and accessible using a tools such as telnet or netcat from the remote agent. In the example above the TCP port number is 54671. If the port is closed then open it on the Bamboo server side for incoming connections from the remote agents.
The property "bamboo.jms.broker.uri" should generally be set to "0.0.0.0:port" (which means listen on all interfaces) unless attempting to bind to a specific network interface.
- After checking the above, re-install or re-start the remote agent.
- Remote agent on Bamboo version 5.3 and higher validates the client port during startup to make diagnosing such problems easier.