Elasticsearch Fails to Start due to Incompatible Java Version

Still need help?

The Atlassian Community is here for you.

Ask the community

This article only applies to Atlassian's server products. Learn more about the differences between cloud and server.

This version of Hipchat Server is no longer supported

This article applies to a version of Hipchat Server which is beyond the Atlassian End of Life policy, and is no longer supported.

When was my version deprecated?

The following versions have been deprecated:

  • Hipchat Server 1.3 (EOL Date: Aug 17, 2017)

The following versions will be deprecated soon:

  • Hipchat Server 2.0 (EOL Date: Jun 17, 2018)
  • Hipchat Server 2.1 (EOL Date: Dec 8, 2018)
  • Hipchat Server 2.2 (EOL Date: May 30, 2019)

You can read more about Atlassian's End of Life policy here

You should upgrade to a more recent version of Hipchat Server as soon as you can to take advantage of new features, and security and bug fixes. If possible, you should also consider deploying Hipchat Data Center instead.

Problem

Elasticsearch, a service running on Hipchat Server, has an incompatibility with OpenJDK v1.7.0_55 and fails to start with the following error appearing in /home/admin/logs/elasticsearch.log/elasticsearch.log:

2015-10-17T13:21:35+00:00 hipchat.example.com [ERROR][bootstrap                ] [hces]Exception
2015-10-17T13:21:35+00:00 hipchat.example.com java.lang.RuntimeException: Java version: 1.7.0_55 suffers from critical bug https://bugs.openjdk.java.net/browse/JDK-8024830 which can cause data corruption.
2015-10-17T13:21:35+00:00 hipchat.example.com Please upgrade the JVM, see http://www.elastic.co/guide/en/elasticsearch/reference/current/_installation.html for current recommendations.
2015-10-17T13:21:35+00:00 hipchat.example.com If you absolutely cannot upgrade, please add -XX:-UseSuperWord to the JVM_OPTS environment variable.
2015-10-17T13:21:35+00:00 hipchat.example.com Upgrading is preferred, this workaround will result in degraded performance.

Diagnosis

Environment

  • Hipchat Server v1.3.x upgraded from a pre-beta deployment

Diagnostic Steps

  • Long-term chat history (more than 75 messages prior to current) may fail to load
  • Hipchat Server exports may fail

Cause

Pre-beta versions of Hipchat Server (~ v1.0.x) came bundled with OpenJDK v1.7.0_55, which is not upgraded as part of Hipchat Server application updates up to Hipchat Server v1.3.5.

Workaround

In an emergency, the following steps can be used to restart Elasticsearch:

Add Parameter Emergency Workaround

  1. Log into the Hipchat Server command line interface
  2. Run the following command to gain root access to the system:
    sudo dont-blame hipchat
  3. Open the /usr/local/etc/elasticsearch/elasticsearch-env.sh file with nano or vim to add "-XX:UseSuperWord" under the "ES_JAVA_OPTS=" section
  4. Run the following command to start Elasticsearch:
    service elasticsearch start


A healthier long-term option, is to patch OpenJDK with the following steps:

tip/resting Created with Sketch.

The following steps require restarting components within Hipchat Server and will result in temporary disruption of service.


Patch to Upgrade OpenJDK

  1. Log into the command-line interface
  2. Run the following command to gain root access to the system: 

     sudo dont-blame-hipchat
  3. Add a new directory for the package download/unpack

    mkdir /tmp/jdk 
  4. Change to the new temporary directory

     cd /tmp/jdk
  5. Download the necessary files for installing the Java

     wget https://hipchat-server-stable.s3.amazonaws.com/patch/openjdk/openjdk-7-7u65-2.5.1-4ubuntu1~0.12.04.2.tar
  6. Unpack the resulting tar file

     tar -xvf openjdk-7-7u65-2.5.1-4ubuntu1~0.12.04.2.tar
  7. Install the downloaded Java packages

     dpkg -i *.deb
  8. Update apt for the installation

     apt-get -f -y install
  9. Restart the database-related services

     hipchat service -u db
  10. Restart the remaining services

     hipchat service -u
  11. Check that Java is up to 1.7.0_65

     java -version

Resolution

Upgrade to the latest version of Hipchat Server to tackle this issue permanently.

Last modified on Nov 14, 2018

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.