Troubleshooting Remote Agents

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Summary

Technical Overview of Remote Agent Connectivity Mechanism

  • The remote agent and server communication is done through HTTP and JMS protocols, on top of separate unicast TCP connections.
  • The remote agent and server communication is bi-directional, and the remote agent initiates both TCP connections.
  • Remote agents use the HTTP protocol while bootstrapping and registering against Bamboo.
  • Artifacts are also transferred using the HTTP protocol.

The JMS protocol is used for other communications (such as dispatching jobs, heartbeat messages, etc) together with the Apache ActiveMQ (AMQ) message broker.

HTTP Connection

The remote agent normally connects to Bamboo using the same URL used for general user access.

Examples of addressesPortContext pathProtocol
http://my.bamboo:8085/agentServer/8085<empty>HTTP
https://my.bamboo.com/bamboo/agentServer/443bambooHTTPS
http://192.168.10.10/bamboo/agentServer/80bambooHTTP

JMS Connection

This connection will be established to the address configured in the <bamboo-local-home>/bamboo.cfg.xml file. There are two properties defined here:

PropertyExplanation
bamboo.jms.broker.client.uriThe broker client URI is the address distributed to the agents. This is the endpoint they will attempt to connect to. 
bamboo.jms.broker.uri

The broker URI is the address Bamboo will use to bind the primary JMS connector.

  • 0.0.0.0:54663 means it will listen on all network interfaces on port 54663.
tip/resting Created with Sketch.
  • Any changes to these properties will require a Bamboo restart.
  • The address should NOT be set to localhost. Ideally, the agent should reach the server directly to its IP or hostname. If a reverse proxy address is set here, please ensure the proxy is correctly configured to redirect the broker TCP traffic.
  • Bamboo also spins up two additional JMS connectors to handle SSL and elastic agent traffic. Bamboo looks at the primary JMS connector port number to determine the port numbers it will use for the additional connectors. This is the formula used by Bamboo:
    • SSL JMS connector = primary JMS connector port number + 1. If the primary JMS connector is 54663, then Bamboo will use 54664 for the SSL JMS connector.
    • Elastic agents JMS connector = primary JMS connector port number + 2. If the primary JMS connector is 54663, then Bamboo will use 54665 for the elastic agents JMS connector. This connector is bound to localhost, by default.
Examples of addressesPortProtocol
tcp://bamboo.hostname:5466354663TCP (JMS)
ssl://192.168.10.10:5466454667TCP (JMS - secure)
nio://bamboo.hostname:5466354663TCP (JMS - buffered TCP)

Diagnosis

The commands below can be used for primary connectivity checks and should succeed if the correct addresses and ports are used.

Agent-side checksCommandExamplesRelated articles
    • Is the agent able to reach and resolve the server hostname?
ping <Server_Hostname>ping my.bamboo.url
    • Is the HTTP server port reachable?
telnet <Server_Hostname> <HTTP_Port>telnet my.bamboo.url 8085
    • Is the agent able to reach and resolve the Broker address?
ping <Server_Hostname or IP_address>

ping 192.168.10.10
ping bamboo.hostname


    • Is the server Broker port reachable?
telnet <Server_Hostname or IP_address> <JMS_Port>

telnet 192.168.10.10 54663
telnet bamboo.hostname 54663


Server-side checks CommandExamples
    • Is the server able to resolve its hostname?
ping <Server_Hostname or IP_address>ping bamboo.hostname
    • Is the HTTP server port referred by the agent bound?

Windows: netstat -an | findstr <HTTP_Port>
Unix Like: netstat -an | grep <HTTP_Port>

netstat -an | findstr 8085
netstat -an | grep 8085

    • Is the server Broker port bound?
Windows: netstat -an | findstr <Broker_Port>
Unix Like: netstat -an | grep <Broker_Port>
netstat -an | findstr 54663
netstat -an | grep 54663

Increasing debug logging

You'll find instructions on how to enable debug logging for Bamboo and/or agents inside the following page:

Click here for a list of classes that can be set to DEBUG inside Bamboo and/or agents...

Here's a list of classes that can be set to DEBUG inside Bamboo and/or agents to help troubleshoot the communication between Bamboo and your agents:

PackageAgent / ServerWhat you will see in the logs
com.atlassian.bamboo.trigger.quartz

SERVER

Log messages for all scheduled Plan triggers.
com.atlassian.utils.process

AGENT SERVER

All external commands started by Bamboo including complete information about environment.
com.atlassian.bamboo.v2.trigger.DependencyChainListener

SERVER

Will provide information about which plans will be considered to run as a result of a build
com.atlassian.bamboo.plugins.stash.events

SERVER

All events coming from Stash to Bamboo (branch creation/deletion, commits).
com.atlassian.bamboo.plan.PlanExecutionManagerImpl

SERVER

Additional information about progress of a build through a build pipeline.
com.atlassian.bamboo.chains.ChainExecutionManagerImpl

SERVER

Additional information about progress of a build through a build pipeline.
com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl

AGENT

Main agent loop.
com.atlassian.bamboo.v2.build.task.InitializeBuild

AGENT

Activities run before regular tasks are run (creating / cleaning working directory)
com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask

AGENT

Source checkout.
com.atlassian.bamboo.build.pipeline.tasks.ExecuteBuildTask

AGENT

Everything before and after running a task.
com.atlassian.bamboo.task.TaskExecutorImpl

AGENT

Running a task
com.atlassian.bamboo.v2.build.queue.BambooMessageConverter

AGENT SERVER

Details about messages sent through JMS.
com.atlassian.bamboo.v2.build.agent.BambooAgentMessageListener

SERVER

Deserialization of agent communication on the server side.
com.atlassian.bamboo.plugins.git.GitCommandProcessor

AGENT SERVER

Output of selected git commands
com.atlassian.bamboo.v2.trigger.DefaultChangeDetectionManager

SERVER

Details about change detection (times, etc)
com.atlassian.bamboo.migration

SERVER

Export / import details
com.atlassian.bamboo.plan.branch.BranchDetectionServiceImpl

SERVER

Additional information about automatic plan branch management
net.schmizz.sshj

AGENT SERVER

SSH/SCP task
org.apache.sshd.client.session

AGENT SERVER

Bamboo -> git server (ie Stash) connection initialization
org.apache.sshd.common.channel

AGENT SERVER

Bamboo -> git server communication (lots of data)
com.atlassian.bamboo.build.expiry

SERVER

Build expiry activity
com.atlassian.bamboo.deployments.expiry

SERVER

Deployment expiry
com.atlassian.bamboo.plugins.ssh

SERVER

SSH Proxy (set to DEBUG)
org.apache.sshd.server.session

SERVER

SSH Proxy (set to ALL). (warning)Very, very verbose, so use with caution and as briefly as possible to avoid flooding the logs.

Solution

Bamboo with a reverse proxy

If Bamboo is running behind a reverse proxy and the remote agent is having trouble connecting with Bamboo (in the first stages of connection - using HTTP protocol) or experiencing artifact transfer problems, it might be worth bypassing the proxy to check if it could be causing the issue. The following document can be used for this purpose:

Other common causes for connectivity issues

Some common reasons for port and socket connectivity issues include:

  • Firewalls
  • Antivirus
  • Forward proxies

Please check the above to see if any are interfering or blocking required ports and connections.

Description

The communication between the remote agent and server is done through two protocols, HTTP and JMS on top of separate unicast TCP connections. The communication between the remote agent and server is bi-directional, and the remote agent initiates both TCP connections. The commands listed can be used for primary connectivity checks and should succeed if the correct addresses and ports are used.

Last modified on Nov 10, 2023

Was this helpful?

Yes
No
Provide feedback about this article

In this section

Powered by Confluence and Scroll Viewport.