OutOfMemory Errors in Bamboo elastic agent

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Problem

Bamboo builds start failing when executed on elastic agents and the following exception is observed in the Bamboo System Error Details page (accessible through the Bamboo UI):

java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.io.UnixFileSystem.resolve(UnixFileSystem.java:108)
	at java.io.File.<init>(File.java:367)
	at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1256)
	...

Diagnosis

Environment

Diagnostic Steps

  • If the build process is running out of memory while executing a Maven or Ant task, please check the "Bamboo builds fail due to java.lang.OutOfMemoryError" knowledge base article as it describes how to workaround the problem and allocate more memory to both ANT_OPTS and MAVEN_OPTS.
  • If you are not running one of the tasks mentioned above, but the elastic agent is still running out of memory, please continue to read.

Cause

The default memory setting for the elastic agent is 256MB. It is necessary to adjust the settings to run a bigger elastic agent with sufficient memory.

Resolution

Third-party apps have also be known to increase the memory requirements for Elastic Agents. Artifactory for Bamboo in particular is one that requires more memory than the default configuration when it is loading. If you are looking to reduce the memory requirements for your elastic agents, rather than increase the limits, removing unneeded third-party apps can reduce the overhead.

For Bamboo 8.0.0 and later versions

If you use a custom Elastic Image that has been created before Bamboo 8.0, you must recreate it with the new version of the atlassian-bamboo-agent-elastic-installer.jar on it in order to use the Elastic Agent supervisor.

Elastic Agent startup scripts, provided in the atlassian-bamboo-elastic-image, use a dedicated variable WRAPPER_PROPERTIES that can carry one or multiple properties and is passed to the Elastic Agent startup command. The WRAPPER_PROPERTIES variable can be set up in an Elastic Image startup script in Bamboo to increase the initial and maximum Java heap size:

Go to Bamboo administration >> Overview >> Elastic Bamboo >> Image configurations page. From there edit the desired elastic image and add the following script to the Instance startup script field:

Linux:

sudo su -c "echo \"export WRAPPER_PROPERTIES='\"-Dwrapper.java.maxmemory=2048\" \"-Dwrapper.java.initmemory=512\"'\" >> /etc/profile.d/bamboo.sh"

Windows:

Option 1:
powershell.exe -Command "[System.Environment]::SetEnvironmentVariable(\"WRAPPER_PROPERTIES\", '\"-Dwrapper.java.maxmemory=2048\" \"-Dwrapper.java.initmemory=512\"', 'Machine')"


(info) The WRAPPER_PROPERTIES variable can also be used to pass other properties to the agent, please see the Elastic Agent supervisor documentation for more details.

For Bamboo versions below 8.0.0

It is possible to change the heap size of the elastic agent using the Instance startup script option located on the Bamboo administration >> Overview >> Elastic Bamboo >> Image configurations page. From there edit the desired elastic image and add the following script to the Instance startup script field:

The example below will bump the Maximum Heap size by replacing the "-Xmx256m" entries with "-Xmx512m". You can extend the commands to add more Java Heap memory options such as modifying the initial heap size "-Xms32m"

Linux:

sed -i 's@.*MARKER.*@sed -i "s/-Xmx256m/-Xmx512m/" $startupScript@' /opt/bamboo-elastic-agent/bin/bamboo-elastic-agent

Windows:

Option 1:

If you have created a custom Windows AMI from scratch without using one of our stock images as a base, our start-up scripts will likely not execute unless you configured it in exactly the same way as our stock images in the Task Scheduler. If startup scripts do not execute, this workaround will not work and you should refer to Option #2.

sed -i "s/-Dimagefiles/-Dbamboo.agent.syncOnly -Dimagefiles/" C:\opt\bamboo-elastic-agent\bin\bamboo-elastic-agent.bat
sed -i 's@cd %%OLD_PWD%%@sed -i \'s/-Xmx256m/-Xmx512m/\' \%%TEMP\%%\\\\bamboo-agent-bootstrap-1.bat\n\%%TEMP\%%\\\\bamboo-agent-bootstrap-1.bat ^>^>bamboo-elastic-agent.out 2^>\^&1\ncd %%OLD_PWD%%@' C:\opt\bamboo-elastic-agent\bin\bamboo-elastic-agent.bat
Option 2:

The below option will override heap sizing for any Java processes that are run after the instance starts up so may not be ideal if you're using anything Java in your builds (e.g. Gradle or Maven)

setx -m _JAVA_OPTIONS "-Xms256m -Xmx512m"


This will increase the maximum elastic agent memory to 512MB. It is possible to adjust the -Xmx setting to allocate more memory to the agent, if necessary.

Last modified on Sep 29, 2022

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.