'Unexpected bytes from remote node' error after upgrading Confluence Data Center to 7.18.1 or later and several other versions
Platform Notice: Data Center Only - This article only applies to Atlassian products on the data center platform.
Confluence fails to start with a "Unexpected bytes from remote node, closing socket" error after upgrading Confluence Data Center to one of the following versions:
- 7.18.1 or later
- 7.17.4 or later 7.17.x version
- 7.16.4 or later 7.16.x version
- 7.15.2 or later 7.15.x version
- 7.14.3 or later 7.15.x version
- 7.13.7 or later 7.13.x version
- 7.4.17 or later 7.4.x version
The following error appears in the atlassian-confluence
2022-06-03 09:42:44,473 WARN [hz.confluence.cached.thread-2] [hazelcast.nio.tcp.TcpIpAcceptor] log [10.9.32.224]:5801 [confcluster1] [3.12.11] com.atlassian.confluence.impl.cluster.hazelcast.interceptor.authenticator.NodeConnectionException: Unexpected bytes from remote node, closing socket com.atlassian.confluence.impl.cluster.hazelcast.interceptor.authenticator.NodeConnectionException: Unexpected bytes from remote node, closing socket at com.atlassian.confluence.impl.cluster.hazelcast.interceptor.authenticator.DefaultClusterJoinManager.checkNodeAuthenticationEnabled(DefaultClusterJoinManager.java:68) at com.atlassian.confluence.impl.cluster.hazelcast.interceptor.authenticator.DefaultClusterJoinManager.accept(DefaultClusterJoinManager.java:47) at com.atlassian.confluence.impl.cluster.hazelcast.interceptor.ClusterJoinSocketInterceptor.onAccept(ClusterJoinSocketInterceptor.java:49) at com.hazelcast.nio.NodeIOService.interceptSocket(NodeIOService.java:300) at com.hazelcast.nio.tcp.TcpIpAcceptor$AcceptorIOThread.configureAndAssignSocket(TcpIpAcceptor.java:316) at com.hazelcast.nio.tcp.TcpIpAcceptor$AcceptorIOThread.access$1400(TcpIpAcceptor.java:138) at com.hazelcast.nio.tcp.TcpIpAcceptor$AcceptorIOThread$1.run(TcpIpAcceptor.java:305) at com.hazelcast.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:227) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at com.hazelcast.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:64) at com.hazelcast.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:80) Collapse
This problem occurs when you attempt to start the second node before the first node has fully started up after the upgrade. You must wait for node 1 to start completely before attempting to start the next node. This particular issue only happens the first time Confluence is started after upgrading to a version listed above.
To resolve this issue, restart the first Confluence node.
We always recommend starting your nodes one at a time, and waiting for each the 'Confluence is ready to serve' message in the atlassian-confluence
.log before attempting to start the next node.