Forking JVM: error=12, Cannot allocate memory or error=12, Not enough space
Symptoms
error=12, Cannot allocate memory (Linux)com.atlassian.bitbucket.exception.ServerException: An error occurred while executing an external process: java.io.IOException: error=12, Cannot allocate memoryerror=12,Not enough space (Solaris)
com.atlassian.bitbucket.exception.ServerException: An error occurred while executing an external process: error=12, Not enough spaceCause
If you received error=12, Cannot allocate memory or error=12, Not enough space, this means your system ran out of memory or swap space when Java tried to fork a process.
The problem is inherent with the way Java allocates memory when executing processes. When Java executes a process, it must fork() then exec(). Forking creates a child process by duplicating the current process. By duplicating the current process, the new child process will request approximately the same amount of memory as its parent process, essentially doubling the memory required. However, this does not mean all the memory allocated will be used, as exec() is immediately called to execute the different code within the child process, freeing up this memory. As an example, when Bitbucket Server tries to locate git, the Bitbucket Server JVM process must be forked, approximately doubling the memory required by Bitbucket Server.
Note that this issue is made worse by hosting multiple webapps in the same Tomcat container as Bitbucket Server (which is not supported), because more memory is used by the Java process.
Please see Forking JVM for a more detailed explanation.
Resolution
If you are hosting multiple products in the same Tomcat container as Bitbucket Server, move Bitbucket Server to its own Tomcat container.
For Linux, this can be resolved by enabling over-committing memory.
For Solaris, we recommend increasing your swap space. How much swap space do you need? This will also depend on the other applications running on the same machine. We recommend allocating at least swap equal to 4 x JVM_MAXIMUM_MEMORY, or 2 x PHYSICAL RAM, whichever is larger. The JVM_MAXIMUM_MEMORY is set in your <Bitbucket Server installation directory>/bin/setenv.sh. You can increase your swap space if you are still running out of memory.
setenv and environment variable changes in Bitbucket Server 5.0+
Starting with Bitbucket Server 5.0, setenv.sh and setenv.bat have been removed. The options that were set in this file can now be set via environment variables. Where to set the environment variable depends on which Operating System you're running on.
Linux
When using the atlbitbucket service on Linux, the environment variables are ignored. You must set the parameters in _start-webapp.sh (or start-bitbucket.sh). These values will be read when the service starts.
As an example, to set JVM_SUPPORT_RECOMMENDED_ARGS, you would add this line to the file:
JVM_SUPPORT_RECOMMENDED_ARGS=-XX:+HeapDumpOnOutOfMemoryErrorWindows
Set the parameter as an environment variable for the user running Bitbucket Server. For example, if you want to set JVM_SUPPORT_RECOMMENDED_ARGS, create it as an environment variable and assign the appropriate value to it. When Bitbucket Server starts using the startup scripts or service, it will pick up and apply this value.