Jira stats logs
Intro
Jira has a number of statistics written to the primary Jira log file (atlassian-jira.log*). These are logs which contain aggregated information of what is happening in Jira, which is expensive to log for every event, but may be very useful when determining how crucial Jira modules (like: indexing, index propagation, node to node communication, DB access, IO) are performing.
By default Jira is logging stats every 5 minutes. These logs may have different formats but share the same concepts:
- total stats - These are statistics gathered from the time the Jira instance was started up. Usually you want to check the last "total" statistics for a given feature to have a general overview of how things are performing
- snapshot - These are the statistics gathered for every snapshot which normally covers a five-minute period (please see below how this time interval can be configured using a new property introduced in Jira 8.16). Usually when you find some values which need more investigation in "total" stats, you will try to find the "snapshot" were the interesting event happened. This will narrow down the search for the problematic event to a five-minute period.
Types of stats logs
Note that Jira stats are being added incrementally since JIRA 8.0 . The list below describes the latest set of available statistics.
It is also important to note that starting with JIRA 8.16 all statistic logs have a common prefix, which can be identified as
[JIRA-STATS].
| Stats prefix | Description | Since | More |
|---|---|---|---|
| [LOCALQ] [VIA-INVALIDATION] | Stats related to replicating messages from caches replicated via invalidation |
| Monitoring the cache replication |
| [LOCALQ] [ VIA-COPY ] | Stats related to replicating messages from caches replicated via copy |
| |
| [DBR] [SENDER] | Stats from the process generating DBR messages. This is just after we produced the document and before we put it into Lucene. And before it sends the message to LocalQ. | JIRA 8.12 | Document-based replication in Jira Data Center |
| [DBR] [RECEIVER] | Stats from the process receiving DBR messages from other nodes though the network. It will store those messages to Lucene right away. | JIRA 8.12 | |
| [VERSIONING] | Stats related to issue, comments and worklog versioning. Measures the impact of the extra DB calls for do the maintenance of the versions for the issue, comments and worklog. | JIRA 8.10 | VERSIONING |
| [index-writer-stats] | Write operations for the indexer (aka Lucene). Placing the created Docs into the Lucene (disk). Jira produces the documents in the different places and places them into the Queue, then single thread process Queue and the writing docs to the index (this will be conditional update). | JIRA 8.10 | |
| [indexing-stats] | Stats focusing on the cost of producing the CustomField values (aka FieldIndexers). | JIRA 8.1 | Indexing Stats - cost of indexing custom fields |
| [lucene-stats] | Measure the Lucene index flush (not related to commit), including Lucene buffer. During foreground indexing index flushes are only caused by Lucene buffer overflow. Opening new searchers requires the flush operation. | JIRA 8.0 | JIRA Indexing Performance and Lucene maxrambuffermb |
| [INDEX-REPLAY] [STATS] | Node Index Replay is the process responsible for replaying index operation from other nodes. | JIRA 8.10 | INDEX-REPLAY STATS |
| [replication-stats] | Stats measuring the latency of replaying index operations from other nodes. | JIRA 8.8 | replication-stats |
| [plugin-transaction] | Stats related to how plugin lifecycle events are triggering cache resets. | JIRA 8.2 | JRASERVER-64908 - Getting issue details... STATUS |
| [OFBIZ-PROPERTY-ENTRY-STORE-STATS] | Stats related to accessing the common property store cache. | JIRA 8.14 | JRASERVER-70519 - Getting issue details... STATUS |
| [INDEX-FIXER] | Stats related to the server component responsible for making sure the index is consistent. | JIRA 8.12 | |
| [INDEXING-QUEUE] | Stats about the state to the indexing queue. This is the part of Jira between the threads building issue (and comments, worklogs, ...) Lucene documents and adding/updating the local index with those documents. | JIRA 8.16 / JIRA 8.13.6 | Jira Indexing Queue |
| [JIRA-RMI-AUTH] | Stats related to the Jira Data Center cluster authentication for node-to-node cache replication RMI connection creation. | JIRA 8.17 / JIRA 8.13.8 / JIRA 8.5.16 | |
| [index-searcher-stats] | Stats related to Lucene's index searchers (searching and creating searchers). | JIRA 8.19 | Jira Stats - search performance [index-searcher-stats] |
| [UserSearchServiceStats] | Stats related to find top mentionable & assignable searcher. | JIRA 8.19.1 | |
[LOGIN-STATS] [LOGIN-STORE-STATS] [GROUP-DAO-STATS] | Stats related to user log-in. | JIRA 8.22.0 | User-login JIRA stats logs |
[INDEXING-LIMITS] | Stats related to safeguards in indexing issue-related entities. | JIRA 8.22.2 / JIRA 9.0.0 | Jira indexing-limits stats |
| [NODE-START] | Stats related index management on startup. | JIRA 9.1.0 | Failed getting index on start |
[AvatarManagerStats] | Avatar stats. | JIRA 9.6.0 | AvatarManagerStats |
| [CacheStats] | Jira-stats for clear cache operations | JIRA 9.6.0 | CacheStats |
| [DVCS-MESSAGE] | Stats related to internal event processing system. Every event is presented as message, placed in queue and consumed. | JIRA 9.8.0 | KB: DVCS Stats - Message |
| [DVCS-HTTP] | Stats related to http traffic which is deducted to two separate streams - inbound and outbound traffic. | JIRA 9.8.0 | KB: DVCS Stats - HTTP |
[DVCS-PERFORMANCE] | Stats related to performance of synchronisation with git data done by DVCS plugin. | JIRA 9.8.0 | KB: DVCS Stats - PERFORMANCE |
Troubleshooting performance with Jira Stat
Please review the related KB article to check how stats-logs (metrics) can be used to troubleshoot different performance problems: Troubleshooting performance with Jira Stats
Statistics Interval Configuration
The below new property was introduced in JIRA 8.16 and allows to change the time the statistics are taken and written to the Jira log files.
We strongly recommend to leave the stats frequency as per its default shown below. Since Jira 8.16 the period can be configured via a system property and changed to: 15, 30 or 60 minutes.
-Dcom.atlassian.jira.stats.logging.interval=interval5min (default)
-Dcom.atlassian.jira.stats.logging.interval=interval15min
-Dcom.atlassian.jira.stats.logging.interval=interval30min
-Dcom.atlassian.jira.stats.logging.interval=interval60min
Note that this system property has to be set on every node and it requires a rolling restart of all nodes.