Hipchat Server /var/log is full

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

 

 

Problem

When Hipchat Server's /var/log/ is full, it will cause some problems including, but not limited to:

  • Hipchat Server is inaccessible:
  • If the Hipchat Server is still accessible, users may encounter problems when authenticating. In the /var/log/hipchat/web.log, for example, the admin may see database connection failures:

    Uncaught Kohana_Database_Exception: There was an error connecting to the database: No such file or directory in file /hipchat-scm/web/system/libraries/Database.php on line 218\n\n0: /hipchat-scm/web/system/libraries/Database.php:1172 -> connect()\n1: 
    

     

Diagnosis

  • Log in to the Hipchat Server's command line and execute the df -h command to check the disk space utilization.
  • Alternatively, viewing the /var/log/hipchat/runtime.log clearly shows that the /var/log filesystem has reached 100% capacity:

    + df -m
    Filesystem                               1M-blocks  Used Available Use% Mounted on
    /dev/sda6                                    11844  6496      4746  58% /
    udev                                          3982     1      3982   1% /dev
    tmpfs                                          799     1       799   1% /run
    none                                             5     0         5   0% /run/lock
    none                                          3993     0      3993   0% /run/shm
    /dev/sda1                                      720    62       622  10% /boot
    /dev/mapper/chat_history_vg-chat_history     10076   422      9143   5% /chat_history
    /dev/mapper/file_store_vg-file_store         40314  1000     37266   3% /file_store
    /dev/sda7                                     3934  3816         0 100% /var/log
    

Cause

The disk partition mounted on /var/log is full

Resolution

Refer to this article: Hipchat Server disk space is full. You can either:

  • Increase the size of the storage volumes per our documentation for Increasing disk capacity
  • Investigate the disk space utilization on the server and remove large files/directories or purge the logs (hipchat logs -p) followed by services restart (hipchat service --restart)

 

Last modified on Nov 2, 2018

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.