Notifications fail due to missing curler service
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the server and data center platforms.
This is for an outdated version of Hipchat Server
This article applies to a version of Hipchat Server which will be deprecated soon. After that period the version will no longer be supported.
You should upgrade to a more recent version of Hipchat Server as soon as you can to take advantage of new features, and security and bug fixes.
Problem
After a server upgrade, mail notifications stop working.
The following appears in the /var/log/chef.log
================================================================================
Error executing action `enable` on resource 'service[curler]'
================================================================================
Chef::Exceptions::Exec
----------------------
/usr/sbin/update-rc.d curler defaults returned 1, expected 0
Resource Declaration:
---------------------
# In /hipchat-scm/chef-repo/cookbooks/curler/recipes/default.rb
23: service 'curler' do
24: action [:enable, :start]
25: supports :status => true, :start => true, :stop => true, :restart => true
26: end
27:
Compiled Resource:
------------------
# Declared in /hipchat-scm/chef-repo/cookbooks/curler/recipes/default.rb:23:in `from_file'
service("curler") do
action [:enable, :start]
supports {:status=>true, :start=>true, :stop=>true, :restart=>true}
retries 0
retry_delay 2
service_name "curler"
pattern "curler"
startup_type :automatic
cookbook_name :curler
recipe_name "default"
end
[2017-05-05T01:55:43+00:00] ERROR: Running exception handlers
[2017-05-05T01:55:43+00:00] ERROR: Exception handlers complete
[2017-05-05T01:55:43+00:00] FATAL: Stacktrace dumped to /usr/local/src/chef-stacktrace.out
[2017-05-05T01:55:43+00:00] ERROR: service[curler] (curler::default line 23) had an error: Chef::Exceptions::Exec: /usr/sbin/update-rc.d curler defaults returned 1, expected 0
[2017-05-05T01:55:09+00:00] INFO: Forking chef instance to converge...
[2017-05-05T01:55:43+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
[2017-05-05T01:55:43+00:00] INFO: Allow custom /etc files
[2017-05-05T01:55:43+00:00] INFO: Re-enabling monitoring
* Starting daemon monitor monit
...done.
Recording last chef run finished at 1493949343
[2017-05-05T01:55:43+00:00] ERROR: Error: Configuration failure
[2017-05-05T01:55:43+00:00] INFO: Releasing chef lock
[2017-05-05T01:55:44+00:00] INFO: Cleaning up consul session 664eab8c-fc3f-e884-1e0e-274b37113a14
curler: unrecognized service
=========
curler does not seem to exist in the systemEnvironment
- Hipchat Server upgraded to version 1.4.x or higher.
Diagnosis
Check if the correct
hc-curlerpackage version is installed in Hipchat Server by executing -dpkg -l | grep hc-curlerCheck if the curler service/process is running by executing -
ps aux | grep curler. Below is an example of a running curler service/process:$ ps aux | grep curler admin 9058 0.0 0.0 10460 932 pts/0 S+ 06:27 0:00 grep --color=auto curler hipchat 26850 0.0 0.4 133288 18556 ? Sl May09 0:00 /hipchat-scm/curler/vendor/virtualenv/bin/python /hipchat/curler/current/vendor/virtualenv/bin/twistd -r epoll --umask=022 --pidfile=/var/run/hipchat/curler.pid --syslog --prefix=curler --facility=168 curler --base-urls=http://localhost:8080/_jobs --job-queue=curler --gearmand-server=localhost:4730 --num-workers=5
Cause
Somehow hc-curler package was not properly installed during the upgrade process, which prevented the service from starting up.
Resolution
Remove and purge existing hc-curler package (if any) on the Hipchat Server and re-install it.
SSH into Hipchat Server's terminal/command-line
Gain root access:
sudo dont-blame-hipchatTo remove and purge existing hc-curler package (if any), run this command from the server's terminal:
dpkg --purge hc-curlerTo re-install the hc-curler package, run this command from the server's terminal:
dpkg -i /file_store/archive/pool/main/h/hc-curler/hc-curler_2017.02.27.212247-774360f_all.debRestart Hipchat services:
hipchat service --restartVerify that curler is running:
ps aux | grep curler