How do I know what retention period is good for my audit log?
With the improved audit log available in Jira 8.8, Confluence 7.5 and Bitbucket 7.0 and later (Server and Data Center), we've introduced a more granular approach to setting the retention period for the audit log that's stored in the database. You can select the exact number of days, months or years you want to keep the log for. The retention period you select is capped at 10 million records.
It's good to have some event history available to monitor what's happening on the instance and spot any trends that might affect stability, or events that might be a possible security threat. However, storing a great number of events might increase the size of the database, and, as a result, decrease the instance’s performance.
Consider the following when deciding on an optimal retention period for your instance:
Server or Data Center
The retention period you might want to pick depends on whether you’re running a Server or a Data Center instance. If you’re running a Data Center instance, you can choose more coverage levels, which can increase the volume of events. However, apart from logging to the database, you can also log events to a file, which helps you collect all the events you need without fearing about your database size growing out of proportions. For more on the auditing feature in Server and Data Center see:
Number of active users
Instances with a high number of active users generate more audit events. On average, 600 active users generate around 20,000 events daily. In this example, the instance would hit the cap of 10 million records in around 1.5 years. If the number of active users were to double, the instance would reach the cap in around 8 months.
Number of events for each coverage area at the selected coverage level
Frequent events affect the overall log volume. Frequent events are usually logged at the Full coverage level however some of them are also logged at Base. It’s a good idea to review the events logged at each level to understand how much volume they’ll generate. For details, see:
3rd party apps
External apps using the REST API can produce a number of events, too. These events and their volume are app-dependant. Consider reviewing the number of 3rd party apps you have and estimate the event volume they might produce.
Auditing and legal constraints
Review any internal and external requirements your organization needs to fulfil regarding the events you need to track and their granularity.
If you need long-term audit log retention, aggregation and management, it may be a good idea to integrate with an external tool, such as Splunk. If you run a Data Center, the audit events are automatically logged to a database and to a file. This option is not available in Server (except for Bitbucket Server which also logs to a file).
Short and long-term solutions
As mentioned above, the number of logged events affect the size and performance of the database, which is why storing the audit log in your database should only be a short-term solution. If you do not tend to log all possible events and your instance is small or medium (150-300 users), the retention period of 3 years seems to be a good start.
However, if you run a large Data Center with over 1000 users, and you want full coverage of your instance, then you're likely to hit the 10 million cap in weeks rather than months. That is why we recommend setting the lowest possible database retention period so that you can quickly view and browse the latest events and use the audit log file to integrate with an external log management tool for long-term storage. For more on the integration capabilities of the audit log, see:
- Audit log integrations in Jira
- Audit log integrations in Confluence
- Audit log integrations in Bitbucket