Bamboo service killed with exception failed with result 'oom-kill'
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
This page covers a scenario where Bamboo service is killed unexpectedly by the Linux OS on which it is hosted.
The problem discussed on this page is related to incorrect heap configurations and it is not a single source resolution for all problems related to 'oom-kill'.
Environment
Applicable for all supported version of Bamboo.
Diagnosis
Bamboo service is unexpectedly killed by Linux OS, there is no errors/warnings in <bamboo-home>logs>atlassian-bamboo.log file. The log file suddenly stops at a certain time.
Inside /var/log folder of Linux OS, we can see below message which means the Bamboo service was killed by the Linux OS kernel.
bamboo.service: Main process exited, code=killed, status=9/KILL
bamboo.service: Failed with result 'oom-kill'.Cause
The Out Of Memory Killer or OOM Killer is a process that the linux kernel employs when the system is critically low on memory. This situation occurs because the linux kernel has over allocated memory to its processes.
When a process starts it requests a block of memory from the kernel. This initial request is usually a large request that the process will not immediately or indeed ever use all of. The kernel, aware of this tendency for processes to request redundant memory, over allocates the system memory. This means that when the system has, for example, 2GB of RAM the kernel may allocate 2.5GB to processes.
Let's explain the cause with an example, Let's say the current Physical memory capacity of the server is 10 GB and you allocate the Heap as -Xms=5120 and -xmx=12288 ( values are in MB, which means 5 and 12 GB respectively ).
Here important point to note is that the capacity of the physical memory is only 10 GB, but max Heap has been assigned as 12GB, you will get a runtime failure when the JVM processes actually tries to use more memory than the machine has, in this case the moment the usage of JVM crosses 10GB.
Solution
We should always define heap size considering how much Physical memory is available on the server, In this case let's say the RAM available is 10GB, -xmx value should not be more than 75% of RAM.
Conservatively keep xmx value capped at 50% of RAM size.
The solution is to correct the configuration of -xmx to a reasonable value.