Unable to receive push notifications and emails on Hipchat Server
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the server and data center platforms.
This is for an outdated version of Hipchat Server
This article applies to a version of Hipchat Server which will be deprecated soon. After that period the version will no longer be supported.
You should upgrade to a more recent version of Hipchat Server as soon as you can to take advantage of new features, and security and bug fixes.
Problem
Users do not receive push notifications nor any emails from Hipchat Server and the following appears in the /var/log/hipchat/gearman.log:
7/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite\n why = selectable.doRead()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 215, in doRead\n return self._dataReceived(data)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 221, in _dataReceived\n rval = self.protocol.dataReceived(data)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/protocols/basic.py", line 571, in dataReceived\n why = self.lineReceived(line)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/http.py", line 454, in lineReceived\n self.handleEndHeaders()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/client.py", line 129, in handleEndHeaders\n m()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/client.py", line 162, in handleStatus_301\n from twisted.internet import ssl\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/ssl.py", line 25, in <module>\n from OpenSSL import SSL\nexceptions.ImportError: No module named OpenSSL\n
2016-06-07T21:43:58.143147+00:00 hipchat curler: [CurlerClient,client] Got job: H:hipchat.testing.com:215
2016-06-07T21:43:58.144571+00:00 hipchat curler: [CurlerClient,client] Got job: H:hipchat.testing.com:216
2016-06-07T21:43:58.145719+00:00 hipchat curler: [HTTPPageGetter,client] Python Error: Unhandled Error\nTraceback (most recent call last):\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/log.py", line 88, in callWithLogger\n return callWithContext({"system": lp}, func, *args, **kw)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/log.py", line 73, in callWithContext\n return context.call({ILogContext: newCtx}, func, *args, **kw)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext\n return self.currentContext().callWithContext(ctx, func, *args, **kw)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext\n return func(*args,**kw)\n--- <exception caught here> ---\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite\n why = selectable.doRead()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 215, in doRead\n return self._dataReceived(data)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 221, in _dataReceived\n rval = self.protocol.dataReceived(data)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/protocols/basic.py", line 571, in dataReceived\n why = self.lineReceived(line)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/http.py", line 454, in lineReceived\n self.handleEndHeaders()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/client.py", line 129, in handleEndHeaders\n m()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/client.py", line 162, in handleStatus_301\n from twisted.internet import ssl\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/ssl.py", line 25, in <module>\n from OpenSSL import SSL\nexceptions.ImportError: No module named OpenSSL\n
2016-06-07T21:43:58.146804+00:00 hipchat curler: [HTTPPageGetter,client] Python Error: Unhandled Error\nTraceback (most recent call last):\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/log.py", line 88, in callWithLogger\n return callWithContext({"system": lp}, func, *args, **kw)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/log.py", line 73, in callWithContext\n return context.call({ILogContext: newCtx}, func, *args, **kw)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext\n return self.currentContext().callWithContext(ctx, func, *args, **kw)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext\n return func(*args,**kw)\n--- <exception caught here> ---\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite\n why = selectable.doRead()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 215, in doRead\n return self._dataReceived(data)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 221, in _dataReceived\n rval = self.protocol.dataReceived(data)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/protocols/basic.py", line 571, in dataReceived\n why = self.lineReceived(line)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/http.py", line 454, in lineReceived\n self.handleEndHeaders()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/client.py", line 129, in handleEndHeaders\n m()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/client.py", line 162, in handleStatus_301\n from twisted.internet import ssl\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/ssl.py", line 25, in <module>\n from OpenSSL import SSL\nexceptions.ImportError: No module named OpenSSL\n
2016-06-07T21:44:00.663300+00:00 hipchat curler: [CurlerClient,client] Got job: H:hipchat.testing.com:217
2016-06-07T21:44:00.665438+00:00 hipchat curler: [HTTPPageGetter,client] Python Error: Unhandled Error\nTraceback (most recent call last):\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/log.py", line 88, in callWithLogger\n return callWithContext({"system": lp}, func, *args, **kw)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/log.py", line 73, in callWithContext\n return context.call({ILogContext: newCtx}, func, *args, **kw)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext\n return self.currentContext().callWithContext(ctx, func, *args, **kw)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext\n return func(*args,**kw)\n--- <exception caught here> ---\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite\n why = selectable.doRead()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 215, in doRead\n return self._dataReceived(data)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 221, in _dataReceived\n rval = self.protocol.dataReceived(data)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/protocols/basic.py", line 571, in dataReceived\n why = self.lineReceived(line)\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/http.py", line 454, in lineReceived\n self.handleEndHeaders()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/client.py", line 129, in handleEndHeaders\n m()\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/web/client.py", line 162, in handleStatus_301\n from twisted.internet import ssl\n File "/hipchat-scm/curler/vendor/virtualenv/local/lib/python2.7/site-packages/twisted/internet/ssl.py", line 25, in <module>\n from OpenSSL import SSL\nexceptions.ImportError: No module named OpenSSL\nDiagnosis
Environment
- Hipchat Server 1.4.x upgraded from 1.3.x
- Hipchat Server 2.0.7 and earlier
Cause
- An alternate runit version of starting gearman is still in place and it's not properly upgraded, having concurrent (or only the old version) gearman services running in the server.
- It's possible that you are affected by HCPUB-1407 - Getting issue details... STATUS
Resolution
To resolve the problem, you will need to upgrade to the latest version of Hipchat Server (Upgrading Hipchat Server) which will clear the old/incorrect gearman entries from your instance.
Alternatively, run the 'Workaround' steps found at HCPUB-1407 - Getting issue details... STATUS
This issue is permanently fixed in Hipchat Server 2.1.3
Last modified on Jan 19, 2018
Powered by Confluence and Scroll Viewport.