Confluence cannot startup due to bad DNS
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Confluence cannot be started up because DNS settings of server is not resolving localhost or hostname.
Environment
Any version of Confluence installed on a Linux server
Diagnosis
Confluence will not be accessible on browser after startup and will show the following stack trace in log files (atlassian-confluence.log)
2020-09-21 15:49:59,336 INFO [Catalina-utility-1] [com.atlassian.confluence.lifecycle] contextInitialized Starting Confluence 6.13.5 [build 7901 based on commit hash 40b45ac0f2e5f9d3dc9afa4612f2408d1270e203] - synchrony version 2.1.0-master-9d112c9d
.
2020-09-21 15:57:22,942 INFO [Catalina-utility-1] [springframework.web.context.ContextLoader] initWebApplicationContext Root WebApplicationContext: initialization completed in 438904 ms
2020-09-21 16:00:03,395 WARN [C3P0PooledConnectionPoolManager[identityToken->1bqoy0xac1cmhljqn2lvk0|450fc2c4]-AdminTaskTimer] [mchange.v2.async.ThreadPoolAsynchronousRunner] log com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@5bfa10a0 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
The above is a symptom of an issue and thread dumps will give more information. A usual stack trace will look like follows where the C3P0
connection manager is trying to connect to the database after resolving localhost.
"C3P0PooledConnectionPoolManager[identityToken->1bqoy0xab1tkj8q5e4x84f|723106d6]-HelperThread-#0" #675036 daemon prio=1 os_prio=0 tid=0x00007f6ab8362800 nid=0xe95f waiting for monitor entry [0x00007f6a7f39f000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.net.InetAddress.getLocalHost(InetAddress.java:1486)
- waiting to lock <0x0000000080ca1240> (a java.lang.Object)
at com.microsoft.sqlserver.jdbc.Util.lookupHostName(Util.java:665)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:2211)
Though, the above thread never finishes and hence Confluence cannot get a connection to the database.
Tomcat logs (catalina.out)
will also show similar issues and we can find other thread stuck on the same stack trace.
21-Sep-2020 12:49:56.303 WARNING [Catalina-utility-3] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadDetected Thread [http-nio-8090-exec-59618] (id=[675001]) has been active for [69,812] milliseconds (since [9/21/20 12:48 PM]) to serve the same request for [http://localhost:8090/synchrony-proxy/v1/bayeux-sync1] and may be stuck (configured threshold for this StuckThreadDetectionValve is [60] seconds). There is/are [30] thread(s) in total that are monitored by this Valve and may be stuck.
java.lang.Throwable
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at java.net.InetAddress.checkLookupTable(InetAddress.java:1394)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1311)
Cause
Lack of name resolution for localhost
or hostname
can cause various issues in Confluence including unable to connect to the database. Simple commands like follows, can be used to verify the cause.
host localhost
host 127.0.0.1
Solution
Ensure that localhost
and hostname
can be resolved properly in Confluence server and use the commands from above section.
- Check
/etc/hosts
- Check
resolv.conf
file