Unable to Connect to SSL Services Due to 'PKIX Path Building Failed' Error in Jira Datacenter and Server

Problem

Attempting to access applications or websites that are encrypted with SSL (for example HTTPS, LDAPS, IMAPS) throws an exception and the connection is refused. This can happen when attempting to establish a secure connection to any of the following:

• Active Directory server, JIRA User Server or Crowd
• Mail server 
• Another Atlassian application using Application Links
• Atlassian Marketplace
• Atlassian Migration Service

For example, the following error appears in the UI when Using the JIRA Issues Macro:

Error rendering macro: java.io.IOException: Could not download: https://siteURL/jira/secure/IssueNavigator.jspa?view=rss&&type=12&type=4&type=3&pid=10081&resolution=1&fixfor=10348&sorter/field=issuekey&sorter/order=DESC&sorter/field=priority&sorter/order=DESC&tempMax=100&reset=true&decorator=none

While the following appears in the logs:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Diagnosis

1. Download SSLPoke.class

2. Execute the class as per the below, changing the URL and port appropriately. Take care that you are running the same Java your application (Confluence, Jira, etc.) is running with. If you used the installer you will need to use <application-home>/jre/java

   $JAVA_HOME/bin/java SSLPoke jira.example.com 443

   A mail server may be mail.example.com 465 

A failed connection would produce the below:

$JAVA_HOME/bin/java SSLPoke jira.example.com 443
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
	at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
	at sun.security.validator.Validator.validate(Validator.java:260)
	at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
	at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
	at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
	at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1351)
	at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:156)
	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:925)
	at sun.security.ssl.Handshaker.process_record(Handshaker.java:860)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1043)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1343)
	at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:728)
	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:138)
	at SSLPoke.main(SSLPoke.java:31)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:145)
	at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:131)
	at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
	at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)
	... 15 more

To get more details from a failed connection, use the -Djavax.net.debug=ssl parameter. For example: 

java -Djavax.net.debug=ssl SSLPoke jira.example.com 443

A successful connection would look like this:

$JAVA_HOME/bin/java SSLPoke jira.example.com 443
Successfully connected

If -Djavax.net.ssl.trustStore is present in your JVM arguments, Java will use the truststore configured instead of the default (cacerts).  You can verify whether the -Djavax.net.ssl.trustStore parameter is causing problems by running the SSLPoke test using the same JVM argument which will execute SSLPoke using your custom truststore. For example:

$JAVA_HOME/bin/java -Djavax.net.ssl.trustStore=/my/custom/truststore -Djavax.net.debug=ssl SSLPoke jira.example.com 443

If this fails (confirming that the truststore doesn't contain the appropriate certificates), the certificate will need to be imported into your defined custom truststore using the instructions in Connecting to SSL Services.

Cause

Whenever Java attempts to connect to another application over SSL (e.g.: HTTPS, IMAPS, LDAPS), it will only be able to connect to applications it can trust.  The way trust is handled in Java is that you have a truststore (typically $JAVA_HOME/lib/security/cacerts). The truststore contains a list of all known Certificate Authority (CA) certificates, and Java will only trust certificates that are signed by one of those CAs or public certificates that exist within that truststore. For example, if we look at the certificate for Atlassian, we can see that the *.atlassian.com certificate has been signed by the intermediate certificates, DigiCert High Assurance EV Root CA and DigiCert High Assurance CA-3. These intermediate certificates have been signed by the root  Entrust.net Secure Server CA :

These three certificates combined are referred to as the certificate chain, and, as they are all within the Java truststore (cacerts), Java will trust any certificates signed by them (in this case, *.atlassian.com). Alternatively, if the *. atlassian.com  certificate had been in the truststore, Java would also trust that site.

This problem is therefore caused by a certificate that is self-signed (a CA did not sign it) or a certificate chain that does not exist within the Java truststore. Java does not trust the certificate and fails to connect to the application.

For details on how to examine a website's certificate chain, see the section, View a certificate, in Secure Website Certificate.

Resolution

1. Make sure you have imported the public certificate of the target instance into the truststore according to the Connecting to SSL Services instructions.
2. Make sure any certificates have been imported into the correct truststore; you may have multiple JRE/JDKs. See How to import a public SSL certificate into a JVM for this.
3. Check to see that the correct truststore is in use. If -Djavax.net.ssl.trustStore has been configured, it will override the location of the default truststore, which will need to be checked.
4. If this error results while integrating with an LDAP server over LDAPS and there is more than one LDAP server, then deselect the Follow referrals option within the LDAP user directory configuration per Connecting to and LDAP Directory.  Optionally, import the SSL certificates from the other LDAP servers into the Confluence truststore.
5. Check if your Anti Virus tool has "SSL Scanning" blocking SSL/TLS. If it does, disable this feature or set exceptions for the target addresses (check the product documentation to see if this is possible.)
6. If connecting to a mail server, such as Exchange, ensure authentication allows plain text.
7. Verify that the target server is configured to serve SSL correctly. This can be done with the SSL Server Test tool.
8. If all else fails, your truststore might be out of date. Upgrade Java to the latest version supported by your application.

More on SSL Poke

Atlassian's SSL Poke source code can be found here.

You can find forked versions of SSL Poke in the community with support for extra features like Java 11, Proxy, etc.

Good examples:

• https://github.com/gebi/sslpoke
• https://gist.github.com/bric3/4ac8d5184fdc80c869c70444e591d3de