Synchrony process killed due to WMIC error


Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Summary

The Synchrony process is killed, at random, usually after Confluence is running for a couple of days. As Confluence fails to restart the Synchrony process, Collaborative Editing stops working and the users can only edit pages after Collaborative Editing is disabled in the UI.

Synchrony can only be restarted by restarting Confluence.

Environment

  • 6.15.10 - but can affect different Confluence versions after Synchrony was implemented (Confluence 6.0)
  • Windows Server

Diagnosis

The Synchrony process suddenly dies, as can be observed in the Synchrony logs:

atlassian-synchrony.log
2020-05-12 03:20:25,853 DEBUG [184:StdOutHandler [E:\Confluence\jre\bin\java.exe]] {"synchrony":{"message":"synchrony.components.watch-pid [warn] PID 3152 has died. Terminating","ns":"synchrony.components.watch-pid","level":"warn"}}

Almost at the same time, Confluence tries to restart Synchrony with no success.  If running Windows Server 2008 R2, you may see an error message like the following:

atlassian-confluence.log
2020-05-12 03:20:26,401 INFO [184:StdOutHandler [E:\Confluence\jre\bin\java.exe]] [plugins.synchrony.bootstrap.DefaultSynchronyProcessManager] startup Starting Synchrony and enabling Collaborative Editing
2020-05-12 03:20:26,869 ERROR [184:StdOutHandler [E:\Confluence\jre\bin\java.exe]] [atlassian.utils.process.ExternalProcessImpl] notifyAfterFinished Error while processing 'afterFinished' event:
org.zeroturnaround.exec.InvalidExitValueException: Unexpected exit value: -2147217402, allowed exit values: [0], executed command [C:\Windows\System32\Wbem\WMIC.exe, process, where, ProcessId=8144, get, ProcessId], output was 66 bytes:
Node - MELP-CONA01

ERROR:

Description = Out of memory
    at org.zeroturnaround.exec.InvalidExitUtil.checkExit(InvalidExitUtil.java:27)
    at org.zeroturnaround.exec.WaitForProcess.call(WaitForProcess.java:114)
    at org.zeroturnaround.exec.ProcessExecutor.executeNoTimeout(ProcessExecutor.java:938)
    at org.zeroturnaround.process.WindowsProcess.isAlive(WindowsProcess.java:78)
    at org.zeroturnaround.process.WindowsProcess.taskkill(WindowsProcess.java:115)
    at org.zeroturnaround.process.WindowsProcess.destroy(WindowsProcess.java:93)
    at org.zeroturnaround.process.AbstractProcess.destroyForcefully(AbstractProcess.java:113)
    at org.zeroturnaround.process.AbstractProcess.destroyForcefully(AbstractProcess.java:16)
    at org.zeroturnaround.process.ProcessUtil.destroyForcefullyAndWait(ProcessUtil.java:114)
    at org.zeroturnaround.process.ProcessUtil.destroyGracefullyOrForcefullyAndWait(ProcessUtil.java:180)
    at com.atlassian.confluence.plugins.synchrony.utils.SynchronyProcessUtil.killRunningSynchronyProcess(SynchronyProcessUtil.java:51)
    at com.atlassian.confluence.plugins.synchrony.bootstrap.DefaultSynchronyProcessManager.startup(DefaultSynchronyProcessManager.java:628)
    at com.atlassian.confluence.plugins.synchrony.bootstrap.DefaultSynchronyProcessManager$SynchronyProcessMonitor.onAfterFinished(DefaultSynchronyProcessManager.java:329)
    at com.atlassian.utils.process.ExternalProcessImpl.notifyAfterFinished(ExternalProcessImpl.java:601)
    at com.atlassian.utils.process.ExternalProcessImpl.wrapUpProcess(ExternalProcessImpl.java:669)
    at com.atlassian.utils.process.ExternalProcessImpl.finish(ExternalProcessImpl.java:192)
    at com.atlassian.utils.process.ExternalProcessImpl$AsynchronousOutputHandlerRunnable.maybeFinish(ExternalProcessImpl.java:730)
    at com.atlassian.utils.process.ExternalProcessImpl$AsynchronousOutputHandlerRunnable.run(ExternalProcessImpl.java:724)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

The Out of Memory message can be misleading, as Synchrony actually has not run out of memory. Increasing the Synchrony heap allocation doesn't solve the issue.

For newer versions of Windows server, this message may not appear though the PID ### has died. Terminating message will still appear.

Cause

The issue is related to an issue with WMIC. 


Workaround

Changing Collaborative Editing Mode to Limited or OFF will prevent the issue.

Confluence 7.5.x or later versions no longer have Limited as an option.

Solution

For Windows Server 2008 R2, please see the link below where a Patch is available:

Customers have also reported success with upgrading to newer versions of Windows Server.


Last modified on Dec 9, 2020

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.