Application crashes due to 'Internal Error (sharedRuntime.cpp:833)' caused by Java 8 bug

Problem

The application crashes with runtime error and the JVM generates a hs_err_pid file with entries similar to the one below:

# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (sharedRuntime.cpp:833), pid=28631, tid=139690008549120
#  fatal error: exception happened outside interpreter, nmethods and vtable stubs at pc 0x00007f0c285e998f
#
# JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /opt/atlassian/bitbucket/3.6.0.0/bin/hs_err_pid28631.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp

Cause

This happens due to a bug in OpenJDK (JDK-8067755 and JDK-8068663) and is triggered by terminating SSL at Tomcat.

Workaround

Option #1

Add the -XX:-UseAESIntrinsics flag to the JVM.

Bitbucket Server: 
Linux
  1. Edit the file <Bitbucket installation directory>/bin/setenv.sh
  2. Add the -XX:-UseAESIntrinsics JVM argument to the "JVM_SUPPORT_RECOMMENDED_ARGS" variable.
  3. Restart Bitbucket Server.
  4. Monitor your instance with the workaround in place. This will keep the instance stable until a new JRE 8 update is released including the fix for this crash. At that time, once the JRE is upgraded, remove the workaround flag provided above.
Windows Service instructions
  1. Open the command window from Start >> Run >> type in 'cmd' >> Enter
  2. cd to the bin directory of your Bitbucket Server installation directory.
  3. Run:

    tomcat8w //ES//AtlassianBitbucket
  4. Click on the Java tab to see the list of current start-up options.
  5. Add -XX:-UseAESIntrinsics on it's own line under Java Options
  6. Restart Bitbucket Server
Jira
Linux
  1. Edit the file <JIRA installation directory>/bin/setenv.sh
  2. Add the -XX:-UseAESIntrinsics JVM argument to the "JVM_SUPPORT_RECOMMENDED_ARGS" variable.
  3. Restart JIRA.
  4. Monitor your instance with the workaround in place. This will keep the instance stable until a new JRE 8 update is released including the fix for this crash. At that time, once the JRE is upgraded, remove the workaround flag provided above.
Windows Service instructions
  1. Please see Setting Properties and Options on Startup for a WindowsService to modify the property listed below.

  2. Add -XX:-UseAESIntrinsics on it's own line under Java Options
  3. Restart JIRA
Confluence
Linux
Follow the instructions at Configuring System Properties.



The flag provided in the workaround above should work however, the original Tomcat thread suggests adding the parameter below instead:

-XX:CompileCommand=exclude,com/sun/crypto/provider/\*.\*

We've had customers succeed by adding the UseAESIntrinsics parameter so only use this option if that one doesn't work.

Option #2

Add a proxy in front of the application to terminate SSL before Tomcat.

 More information about this can be found here:  

Resolution

We have confirmed that Java 8u60 contains the fix.

Upgrade to Java 8u60 or later.


Last modified on Jan 30, 2018

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.