Confluence Data Center is crashing during startup because the AVX instruction set flag is missing on the Linux server

Platform Notice: Data Center - This article applies to Atlassian products on the Data Center platform.

Note that this knowledge base article was created for the Data Center version of the product. Data Center knowledge base articles for non-Data Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Summary

Unable to start Confluence Data Center on a Linux machine without the AVX instruction set flag

Environment

  • Confluence Data Center 8.8.x to latest.
  • Linux Server.

Diagnosis

  • During the startup, the following JVM crash stack can be seen in the Catalina.out file.
    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  SIGILL (0x4) at pc=0x00007f2af2e49fb4, pid=118018, tid=118041
    #
    # JRE version: OpenJDK Runtime Environment Temurin-17.0.8.1+1 (17.0.8.1+1) (build 17.0.8.1+1)
    # Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.8.1+1 (17.0.8.1+1, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
    # Problematic frame:
    # C  [libbc-lts-avx.so+0x5fb4]  init_bytearray_ctx+0x4
    #
    # Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e" (or dumping to /data/jira/Confluence/confluence_5_4_4/bin/core.118018)
    #
    # An error report file with more information is saved as:
    # /data/jira/Confluence/confluence_5_4_4/bin/hs_err_pid118018.log
    #
    # If you would like to submit a bug report, please visit:
    #   https://github.com/adoptium/adoptium-support/issues
    # The crash happened outside the Java Virtual Machine in native code.
    # See problematic frame for where to report the bug.
    #
  • hs_err_pid logs has the following entries related to the crash.
    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  SIGILL (0x4) at pc=0x00007f007c43b5b4, pid=1594195, tid=1594218
    #
    # JRE version: OpenJDK Runtime Environment Temurin-17.0.8.1+1 (17.0.8.1+1) (build 17.0.8.1+1)
    # Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.8.1+1 (17.0.8.1+1, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
    # Problematic frame:
    # C  [libbc-lts-avx.so+0x85b4]  init_bytearray_ctx+0x4
    #
    # Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e" (or dumping to /data/jira/Confluence/confluence_5_4_4/bin/core.1594195)
    #
    # If you would like to submit a bug report, please visit:
    #   https://github.com/adoptium/adoptium-support/issues
    # The crash happened outside the Java Virtual Machine in native code.
    # See problematic frame for where to report the bug.
    #
    ---------------  S U M M A R Y ------------
    Command Line: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED -Djava.util.logging.config.file=/data/jira/Confluence/confluence_5_4_4/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Datlassian.plugins.startup.options= -Dindex.queue.thread.count=10 -Djdk.tls.server.protocols=TLSv1.1,TLSv1.2 -Djdk.tls.client.protocols=TLSv1.1,TLSv1.2 -Dorg.apache.tomcat.websocket.DEFAULT_BUFFER_SIZE=32768 -Dconfluence.context.path= -Djava.locale.providers=JRE,SPI,CLDR -Dsynchrony.enable.xhr.fallback=true -Datlassian.plugins.enable.wait=300 -Djava.awt.headless=true -Xlog:gc*:file=/data/jira/Confluence/confluence_5_4_4/logs/gc-%t.log:tags,time,uptime,level:filecount=5,filesize=2M -XX:G1ReservePercent=20 -XX:+UseG1GC -XX:+ExplicitGCInvokesConcurrent -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.jar=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.http=ALL-UNNAMED --add-opens=java.base/sun.net.www.protocol.https=ALL-UNNAMED --add-opens=java.base/sun.util.locale=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.management/javax.management=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED --add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED --add-exports=java.base/sun.security.action=ALL-UNNAMED --add-exports=java.base/sun.util.calendar=ALL-UNNAMED --add-exports=java.naming/com.sun.jndi.ldap=ALL-UNNAMED --add-exports=java.xml/com.sun.org.apache.xml.internal.utils=ALL-UNNAMED --add-exports=java.desktop/sun.font=ALL-UNNAMED --add-exports=java.base/sun.security.util=ALL-UNNAMED --add-exports=java.base/jdk.internal.ref=ALL-UNNAMED --add-modules=java.se -XX:ReservedCodeCacheSize=256m -Xms8192m -Xmx8192m -Dignore.endorsed.dirs= -Dcatalina.base=/data/jira/Confluence/confluence_5_4_4 -Dcatalina.home=/data/jira/Confluence/confluence_5_4_4 -Djava.io.tmpdir=/data/jira/Confluence/confluence_5_4_4/temp org.apache.catalina.startup.Bootstrap start
    Host: Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, 4 cores, 15G, Oracle Linux Server release 8.10
    Time: Tue Jul 16 17:22:59 2024 IST elapsed time: 69.342502 seconds (0d 0h 1m 9s)
    ---------------  T H R E A D  ---------------
    Current thread (0x00007f0104f33850):  JavaThread "Catalina-utility-1" [_thread_in_native, id=1594218, stack(0x00007f00d8fb4000,0x00007f00d90b5000)]
    Stack: [0x00007f00d8fb4000,0x00007f00d90b5000],  sp=0x00007f00d90b0da8,  free space=1011k
    Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
    C  [libbc-lts-avx.so+0x85b4]  init_bytearray_ctx+0x4
    Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
    j  org.bouncycastle.crypto.engines.AESNativeCBC.init(J[B[B)V+0
    j  org.bouncycastle.crypto.engines.AESNativeCBC.init(Lorg/bouncycastle/crypto/params/KeyParameter;)V+117
    j  org.bouncycastle.crypto.engines.AESNativeCBC.init(ZLorg/bouncycastle/crypto/CipherParameters;)V+85
    j  org.bouncycastle.crypto.paddings.PaddedBufferedMultiBlockCipher.init(ZLorg/bouncycastle/crypto/CipherParameters;)V+43
    j  org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher$BufferedGenericBlockCipher.init(ZLorg/bouncycastle/crypto/CipherParameters;)V+6
    j  org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher.engineInit(ILjava/security/Key;Ljava/security/spec/AlgorithmParameterSpec;Ljava/security/SecureRandom;)V+1999
    j  org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher.engineInit(ILjava/security/Key;Ljava/security/AlgorithmParameters;Ljava/security/SecureRandom;)V+63
    j  javax.crypto.Cipher.init(ILjava/security/Key;Ljava/security/AlgorithmParameters;Ljava/security/SecureRandom;)V+35 java.base@17.0.8.1
    j  javax.crypto.Cipher.init(ILjava/security/Key;Ljava/security/AlgorithmParameters;)V+7 java.base@17.0.8.1
    j  com.atlassian.db.config.password.ciphers.algorithm.AlgorithmCipher.encrypt(Lcom/atlassian/db/config/password/ciphers/algorithm/paramters/EncryptionParameters;)Lcom/atlassian/db/config/password/ciphers/algorithm/paramters/DecryptionParameters;+49
    j  com.atlassian.db.config.password.ciphers.algorithm.AlgorithmCipher.encrypt(Ljava/lang/String;)Ljava/lang/String;+28
    j  com.atlassian.crowd.crypto.ClusterAwareCipherWrapper.encrypt(Lcom/atlassian/db/config/password/ciphers/algorithm/paramters/EncryptionParameters;)Lcom/atlassian/db/config/password/ciphers/algorithm/paramters/DecryptionParameters;+115
    j  com.atlassian.crowd.crypto.DbConfigPasswordCipherEncryptor.doEncrypt(Ljava/lang/String;Ljava/lang/String;)Lcom/atlassian/db/config/password/ciphers/algorithm/paramters/DecryptionParameters;+83
    j  com.atlassian.crowd.crypto.DbConfigPasswordCipherEncryptor.changeEncryptionKey()Z+4
    j  com.atlassian.crowd.crypto.CachedEncryptor.changeEncryptionKey()Z+4
    j  com.atlassian.crowd.crypto.SaltingEncryptor.changeEncryptionKey()Z+4
    j  com.atlassian.crowd.crypto.MissingKeyHandlingEncryptor.encrypt(Ljava/lang/String;)Ljava/lang/String;+30
    j  com.atlassian.crowd.crypto.PrefixBasedSwitchableEncryptor.encrypt(Ljava/lang/String;)Ljava/lang/String;+75
    j  com.atlassian.crowd.crypto.DirectoryPasswordsEncryptor$$Lambda$2585+0x00007f0086062e38.apply(Ljava/lang/Object;)Ljava/lang/Object;+8
    j  com.atlassian.crowd.directory.LazyAttributesEvaluationDirectory.lambda$extractFromDelegateAndTransform$1(Ljava/lang/String;)Ljava/util/Optional;+41
    j  com.atlassian.crowd.directory.LazyAttributesEvaluationDirectory$$Lambda$2591+0x00007f0086063dd0.call()Ljava/lang/Object;+8
  • When installing Confluence Data Centre 8.8.x with the default internal user directory, there are no startup issues on the machine.
  • The issue starts once we add an external user directory.
  • However, startup issue has been noticed in the 9.x version with internal user directory.

Cause

  • Confluence is crashing while encrypting the directory password.
  • The UserDirectoryPasswordEncryptionUpgradeTask is a newly introduced feature for encrypting LDAP directory passwords, available starting from version 8.8.0.
  • The issue is related to the VMware/hardware configuration. The AVX instruction set flag, used by Bouncy Castle for encryption, is not available on the system, which is causing the problem.
  • The lscpu command will display the CPU flags, allowing us to check whether the AVX flag is available on the machine. AVX flag is not available in the below example.
    [admin@xx.xx.xx.xx ConfluenceHost]$ lscpu
    Architecture:        x86_64
    CPU op-mode (s):     32-bit, 64-bit
    Byte Order:          Little Endian
    CPU(s):              4
    On-line CPU(s) list: 0-3
    Thread(s) per core:  1
    Core (s) per socket: 1
    Socket(s):           4
    NUMA node (s):       1
    Vendor ID:           GenuineIntel
    CPU family:          6
    Model:               85
    Model name:          Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
    Stepping:            4
    CPU MHz:             2294.609
    BogoMIPS:            4589.21
    Hypervisor vendor:   VMware
    Virtualization type: full
    Lld cache:           32K
    Lli cache:           32K
    L2 cache:            1024K
    L3 cache:            25344K
    NUMA node0 CPU(s):   0-3
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpelgb rdtscp lm consta nt_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline _timer aes fl6c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmil smep bmi2 invpcid r seed adx smap clflushopt clwb arat ospke md clear flush lld arch capabilities

Solution

To fix this problem, follow the below steps:

  • Please inform the VMware team to add AVX flag.
  • Below is example of lscpu output with AVX flag (You will be able to see avx and avx2 in the flags options)
    admin@xx.xx.xx.xx :~$ lscpu
    Architecture:                         x86_64
    CPU op-mode(s):                       32-bit, 64-bit
    Byte Order:                           Little Endian
    Address sizes:                        48 bits physical, 48 bits virtual
    CPU(s):                               2
    On-line CPU(s) list:                  0,1
    Thread(s) per core:                   2
    Core(s) per socket:                   1
    Socket(s):                            1
    NUMA node(s):                         1
    Vendor ID:                            AuthenticAMD
    CPU family:                           25
    Model:                                1
    Model name:                           AMD EPYC 7R13 Processor
    Stepping:                             1
    CPU MHz:                              2649.998
    BogoMIPS:                             5299.99
    Hypervisor vendor:                    KVM
    Virtualization type:                  full
    L1d cache:                            32 KiB
    L1i cache:                            32 KiB
    L2 cache:                             512 KiB
    L3 cache:                             4 MiB
    NUMA node0 CPU(s):                    0,1
    Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch topoext invpcid_single ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr rdpru wbnoinvd arat npt nrip_save vaes vpclmulqdq rdpid
  • You should now be able to start Confluence.


In case of further assistance please reach out to the Atlassian Support team and share the information that's been gathered so far along with a fresh Support zip file.


Last modified on Mar 14, 2025

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.