monit scissortail alerts are received after upgrade from 1.1.x to 2.x

Still need help?

The Atlassian Community is here for you.

Ask the community

This article only applies to Atlassian's server products. Learn more about the differences between cloud and server.

This is for an outdated version of Hipchat Server

 This article applies to a version of Hipchat Server which will be deprecated soon. After that period the version will no longer be supported.

When will my version be deprecated?

The following versions have been deprecated:

  • Hipchat Server 1.3 (EOL Date: Aug 17, 2017)
  • Hipchat Server 2.0 (EOL Date: Jun 17, 2018)
  • Hipchat Server 2.1 (EOL Date: Dec 8, 2018)

The following version will be deprecated soon:

  • Hipchat Server 2.2 (EOL Date: May 30, 2019)

You can read more about Atlassian's End of Life policy here.

You should upgrade to a more recent version of Hipchat Server as soon as you can to take advantage of new features, and security and bug fixes.

Problem

After a successful upgrade from 1.1.x to 2.x, Hipchat Server is sending monitoring alerts similar to the following: 

Here are the details,
Action: alert
Desc: failed to start (exit status -1) – Program /hipchat/scissortail/current/wrapper.sh failed: File '/hipchat/scissortail/current/wrapper.sh' does not exist
Event: Execution failed
Service: scissortail-rqworker
Host: hipchat.example.com
Date: Thu, 02 Mar 2017 13:38:52

Here are the details,
Action: alert
Desc: failed to start (exit status 0) – /etc/init.d/scissortail: Traceback (most recent call last):
File "/hipchat-scm/scissortail/vendor/virtualenv/lib/python3.5/site-packages/gunicorn/app/base.py", line 92, in get_config_from_filename
execfile_(filename, cfg, cfg)
File "/hipchat-scm/s
Event: Execution failed
Service: scissortail-gunicorn
Host: hipchat.example.com
Date: Thu, 02 Mar 2017 13:39:23

Looking at /var/log/hipchat/runtime.log or its rotated version /var/log/hipchat/runtime.log.1.gz, you will see

/etc/monit.d/scissortail.monitrc:2: Program does not exist: '/hipchat/scissortail/current/wrapper.sh'
/etc/monit.d/scissortail.monitrc:3: Program does not exist: '/hipchat/scissortail/current/wrapper.sh'

Diagnosis

Validate the symptoms by checking the presence of /etc/monit.d/scissortail.monitrc.

Cause

The upgrade doesn't properly cleanup /etc/monit.d/scissortail.monitrc. This config file tells monit to monitor for a process that no longer exists in Hipchat Server. 

When monit doesn't find this process running, it will try to restart it and will fail. 

Workaround

  1. Move the monit config file out of the way: 

    sudo dont-blame-hipchat
    mv -v /etc/monit.d/scissortail.monitrc /home/admin/scissortail.monitrc.bak # Pick any other location if needed
  2. Restart monit:

    service monit restart
  3. Verify monit doesn't return any process as failed:

    monit status

Resolution

The issue is logged as a bug in our public issue tracker:  HCPUB-2779 - Getting issue details... STATUS

 

Last modified on Jan 19, 2018

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.