Problem

When Hipchat Server's /var/log/ is full, it will cause some problems including, but not limited to:

• Hipchat Server is inaccessible:
  

• If the Hipchat Server is still accessible, users may encounter problems when authenticating. In the /var/log/hipchat/web.log, for example, the admin may see database connection failures:

  Uncaught Kohana_Database_Exception: There was an error connecting to the database: No such file or directory in file /hipchat-scm/web/system/libraries/Database.php on line 218\n\n0: /hipchat-scm/web/system/libraries/Database.php:1172 -> connect()\n1: 
  

   

Diagnosis

• Log in to the Hipchat Server's command line and execute the df -h command to check the disk space utilization.

• Alternatively, viewing the /var/log/hipchat/runtime.log clearly shows that the /var/log filesystem has reached 100% capacity:

  + df -m
  Filesystem                               1M-blocks  Used Available Use% Mounted on
  /dev/sda6                                    11844  6496      4746  58% /
  udev                                          3982     1      3982   1% /dev
  tmpfs                                          799     1       799   1% /run
  none                                             5     0         5   0% /run/lock
  none                                          3993     0      3993   0% /run/shm
  /dev/sda1                                      720    62       622  10% /boot
  /dev/mapper/chat_history_vg-chat_history     10076   422      9143   5% /chat_history
  /dev/mapper/file_store_vg-file_store         40314  1000     37266   3% /file_store
  /dev/sda7                                     3934  3816         0 100% /var/log
  

Cause

The disk partition mounted on /var/log is full

Resolution

Refer to this article: Hipchat Server disk space is full. You can either:

• Increase the size of the storage volumes per our documentation for Increasing disk capacity

• Investigate the disk space utilization on the server and remove large files/directories or purge the logs (hipchat logs -p) followed by services restart (hipchat service --restart)