Administering Jira applications Support

This article describes how to expose JMX MBeans within Jira for monitoring with a JMX client and how to use JMX MBeans for in-product diagnostics.

This guide provides a basic introduction to the JMX interface and is provided as is. Our support team can help you troubleshoot a specific Jira problem but can't help you set up your monitoring system or interpret the results.

On this page:

This article describes how to expose JMX MBeans within Jira for monitoring with a JMX client and how to use JMX MBeans for in-product diagnostics.

What is JMX?

JMX (Java Management Extensions) is a technology for monitoring and managing Java applications. JMX uses objects called MBeans (Managed Beans) to expose data and resources from your application. For large instances of Jira Server or Jira Data Center, enabling JMX allows you to more easily monitor the consumption of application resources and diagnose performance issues related to indexing. This enables you to make better decisions about how to maintain and optimize machine resources.

Details of the JMX metrics available in Jira

JMX attribute name	Description
50thPercentile	The value at the 50th percentile in the distribution of measured times (the median value)
75thPercentile	The value at the 75th percentile in the distribution of measured times
95thPercentile	The value at the 95th percentile in the distribution of measured times
98thPercentile	The value at the 98th percentile in the distribution of measured times
99thPercentile	The value at the 99th percentile in the distribution of measured times
999thPercentile	The value at the 99.9th percentile in the distribution of measured times
Count	The number of invocations since node startup
DurationUnit	The time unit used to report percentile values, min, max, mean and standard deviation. The default unit is milliseconds.
FifteenMinuteRate	The fifteen-minute moving average rate of invocations since node startup. This rate uses the same exponential decay factor as is used for the fifteen-minute load average in Unix’s top command.
FiveMinuteRate	The five-minute moving average rate of invocations since node startup. This rate uses the same exponential decay factor as is used for the five-minute load average in Unix’s top command.
Max	The highest measured time since node startup
Mean	The mean measured time since node startup
MeanRate	The mean rate of invocations since node startup
Min	The lowest measured time since node startup
OneMinuteRate	The one-minute moving average rate of invocations since node startup. This rate uses the same exponential decay factor as is used for the one-minute load average in Unix’s top command.
RateUnit	The unit in which MeanRate, OneMinuteRate, FiveMinuteRate, and FifteenMinuteRate are reported. The default unit is events/second.
StdDev	The standard deviation in the measured times since node startup

Metrics collected by Jira

The following table lists metrics (MBeans) that are collected by Jira. All of them are grouped in the com.atlassian.jira property.

Metric	Description	Reset after restarting Jira
dashboard.view.count	The number of times all dashboards were viewed by users	Yes
entity.attachments.total	The number of attachments	N/A
entity.components.total	The number of components	N/A
entity.customfields.total	The number of custom fields	N/A
entity.filters.total	The number of filters	N/A
entity.groups.total	The number of user groups	N/A
entity.issues.total	The number of issues	N/A
entity.users.total	The number of users	N/A
entity.versions.total	The number of versions created	N/A
issue.assigned.count	The number of times issues were assigned or reassigned to users (counts each action)	Yes
issue.created.count	The number of issues that you created after starting your Jira instance	Yes
issue.link.count	The number of issue links created after starting your Jira instance	Yes
issue.search.count	The number of times you searched for issues	Yes
issue.updated.count	The number of times you updated issues (each update after adding or changing some information)	Yes
issue.worklogged.count	The number of times you logged work on issues	Yes
jira.license	The types of licenses you have, the number of active users, and the maximum number of users available for each license type	N/A
quicksearch.concurrent.search	The number of concurrent searches that are being performed in real-time by using the quick search. You can use it to determine whether the limit set for concurrent searches is sufficient or should be increased.	Yes
web.requests	The number of requests (invocation.count) and the total response time (total.elapsed.time)	Yes

The following metrics have been added to Jira as of 8.1 and are exposed under com.atlassian.jira/metrics. All the following metrics will be reset after restarting Jira.

Metric path	Description
comment	Metrics related to comment operations
comment/Create	A comment being created
comment/Delete	A comment being deleted
comment/Update	A comment being updated
indexing	Metrics related to issue, comment, worklog, and change indexing
indexing/CreateChangeHistoryDocument	Index documents being created for Change History entities. Note that many Change History Documents can be created for every issue.
indexing/CreateCommentDocument	Index documents being created for Comment entities
indexing/CreateIssueDocument	Index documents being created for Issue entities
indexing/IssueAddFieldIndexers	FieldIndexer modules enrich Issue documents as part of Index document creation. Plugins can register custom FieldIndexer modules. These metrics provide insight into how much time is spent in FieldIndexer and can be used to track down indexing performance issues caused by them. The metrics describe how much time was spent in all FieldIndexers combined per Issue document created.
indexing/LuceneAddDocument	How much time was spent adding a document to the Lucene index
indexing/LuceneDeleteDocument	How much time was spent deleting one or more documents matching a term from the Lucene index
indexing/LuceneOptimize	Metrics about Lucene index optimization (triggered manually from Jira)
indeding/LuceneUpdateDocument	How much time was spent adding a created document to the Lucene index
indexing/ReplicationLatency	Replication latency is the time between an issue, comment, or worklog being indexed on the node where the change was made and the indexing operation being replayed on the current node
indexing/WaitForLucene	Documents are written to the Lucene index asynchronously. This metric captures how much time Jira’s indexing thread spent waiting for Lucene to complete the write.
indexing/issueAddSearchExtractors	EntitySearchExtractor enriches issue documents as part of Index document creation. Plugins can register custom EntitySearchExtractor modules. These metrics provide insight into how much time is spent in EntitySearchExtractor and can be used to track down indexing performance issues they cause. The metrics describe how much time was spent in all EntitySearchExtractors combined per issue document created.
issue	Metrics about issue operations
issue/Create	Issue being created
issue/Delete	Issue being deleted
issue/Index	An issue being added to the Lucene index. This covers issue document creation and adding the document to the index.
issue/DeIndex	An issue being removed from the Lucene index
issue/ReIndex	An issue being re-indexed as a result of issue updates. This covers: creating an issue document, deleting the old document from the index, and adding new documents to the index.
issue/Update	An issue being updated

Metrics collected by Assets in Jira Service Management

The following table lists metrics (MBeans) that are collected by Assets in Jira Service Management.

Metric	Description
assets.objectindeximpl.objects_load_on_add_ms	How long to load objects when a message is received. This metric tells you how long it took to populate the index from the database.
assets.objectindeximpl.missing_objects_reload_retry_count_on_add	The number of attempts to reload an object when adding the object to the database. The metric gives an indication on how many attempts it took to load the object from the database due to the delays in writing to the database.
assets.objectindeximpl.missing_objects_count_on_add	The number of missing objects on the first attempt at adding them to the database. This metric indicates the number of missing objects.
assets.objectindeximpl.missing_objects_reload_on_add_ms	The time in milliseconds that was spent trying to reload from the database when an object wasn’t available. This metric shows how much time was spent looping and waiting for the object to be available in the database.
assets.objectindeximpl.objects_load_on_update_ms	The time in milliseconds to load updates for on the first attempt. `onUpdate(final int id, final long updateTime)`
assets.objectindeximpl.missing_objects_reload_on_update_ms	The time in milliseconds that is spent waiting for updates to appear in the database. `reloadObjectFromDatabaseUntilUpdateTimeMatches`
assets.objectindeximpl.missing_objects_reload_retry_count_on_update	The number of attempts to reload an object on update until the right version was found.
assets.objectindeximpl.missing_objects_reload_retry_count_on_update	The number of missing object updates that were not ready in the database when an update started.
assets.objectindeximpl.objects_removal_ms	The time in milliseconds that indicates how long it took to remove objects from Assets. `onRemove(final Collection<Integer> ids)`
assets.assetsbatchreplicationmessageworkqueuepoller.process_ms	The number of milliseconds that indicates how long to process a replication message.
assets.assetsbatchreplicationmessageworkqueuepoller.number_of_create_failures	The number of failures in a message batch when an object is created.
assets.assetsbatchreplicationmessageworkqueuepoller.number_of_update_failures	The number of failures in a message batch when an object is updated.
assets.cachemessageworkqueuepoller.process_batch_size	The number of object changes that were batched together to be sent across the cluster.
assets.cachemessageworkqueuepoller.wait_for_clearance_ms	The time in milliseconds that is spent waiting for database updates to be drained from the application or submitted to the database.
assets.cachemessageworkqueuepoller.process_ms	The time in milliseconds that is the total duration of the process of batching and sending a replication message.
assets.assetsbatchreplicationmessagereceiver.work_queue_size	The size of the work queue in the replication message receiver.
assets.assetsbatchreplicationmessagereceiver.work_queue_gauge	The current size of the work queue in the replication message receiver
assets.assetsobjectreplicationbatchmanager.work_queue_size	The work queue size in the batch manager collecting individual changes to batch and send across the cluster.
assets.assetsobjectreplicationbatchmanager.work_queue_gauge	The current work queue size in the batch manager collecting individual changes to batch and send across the cluster
assets.defaultassetsbatchmessagesender.send_message	The amount of time to dispatch the message and continue processing the next message.
assets.insightcachereplicatorimpl.legacy_object_receiver_queue_size	The queue size on the legacy replication mechanism using cluster messages.
assets.insightcachereplicatorimpl.object_replication_dispatch	The amount of time it takes to `handleMessage` or dispatch a message by calling send for the legacy index replication using the cluster message cache.
assets.insightcachereplicatorimpl.legacy_object_send_queue_size	The queue size after offering a message for legacy index replication using the cluster message cache. `messageToSend.offer(cacheMessage)`
assets.assetsreplicationretryqueuepoller.retry_queue_size	The size of the retry/failure queue.
assets.assetsreplicationretryqueuepoller.create_retry_attempts	The number of attempts to get successful processing for creating a message.
assets.assetsreplicationretryqueuepoller.update_retry_attempts	The number of attempts to get successful processing for updating a message.
assets.assetsreplicationretryqueuepoller.replay_wait_time	The amount of waiting time that was added before retrying the update. This metric helps to see if processing is keeping up with the queue, or if updates are backlogged and being processed immediately. If there is no waiting time applied, the queue is not processed quickly enough.
assets.assetsreplicationretryqueuepoller.dead_letter_queue_size	The number of items in the dead letter queue.
assets.assetsreplicationretryqueuepoller.process_retry_excluding_wait_ms	The time in milliseconds that it takes to process the failures, excluding any wait time prior to processing. This metric is an indication of how long the batches of failures are taking to be processed. It will also help to see if more delays are being added in the `onAdd` or `onUpdate` wait loops. The value of this metric should be low, unless more retries are needed, in which case the delay could be increased.
assets.assetsreplicationretryqueuepoller.dead_letter_queue_gauge	The current number of messages in the dead letter queue.
assets.assetsreplicationretryqueuepoller.retry_queue_gauge	The current number of messages in the retry queue.

Monitoring Jira

Before you can monitor Jira, you should enable JMX monitoring and then use a JMX client to view the metrics.

Good to know

Viewing the metrics will always have some performance impact on Jira. We recommend that you don't refresh them more than once a second.

Enabling JMX monitoring in Jira

All of the metrics are collected by default, but you should enable JMX monitoring to expose them. You can do it in Jira but you must be a Jira admin.

From the top navigation bar select Administration > System.
Go to JMX monitoring.
Toggle Enable JMX monitoring.

Monitoring with JConsole

After you enabled JMX monitoring, you can use any JMX client to view the metrics. To make it quick and easy, we've described how to view them by using JConsole. You can monitor your Jira instance either locally or remotely:

Monitoring Jira locally is good if you're troubleshooting a particular issue or only need to monitor Jira for a short time. Local monitoring can have a performance impact on your server, so it's not recommended for long-term monitoring of your production system.
Show me how to do this...
To monitor locally:
1. Start JConsole. You'll find it in the bin directory of the JDK installation directory. (JConsole is only available as part of the JDK.)
2. Select Local Process.
3. Select the Jira process. It'll be called something like org.apache.catalina.startup.Bootstrap start
4. After connecting, expand the com.atlassian.jira property that groups all the metrics.
See Using JConsole for more information on local monitoring.
Monitoring Jira remotely is recommended for production systems as it does not consume resources on your Jira server.
Show me how to do this...
To monitor remotely:
1. Add the following properties to your setenv.sh / setenv.bat file. The port can be any port that is not in use.
  
  CATALINA_OPTS
  
  Windows
  
  SET CATALINA_OPTS="-Dcom.sun.management.jmxremote.authenticate=true ${CATALINA_OPTS}" SET CATALINA_OPTS="-Dcom.sun.management.jmxremote.password.file=/atlassian-jira-software-x.y.z-standalone/jmxremote.password ${CATALINA_OPTS}" SET CATALINA_OPTS="-Dcom.sun.management.jmxremote.access.file=/atlassian-jira-software-x.y.z-standalone/jmxremote.access ${CATALINA_OPTS}"
  
  x.y.z stands for the Jira version you're using.
  
  Linux
  
  CATALINA_OPTS="-Dcom.sun.management.jmxremote.authenticate=true ${CATALINA_OPTS}" CATALINA_OPTS="-Dcom.sun.management.jmxremote.password.file=/atlassian-jira-software-x.y.z-standalone/jmxremote.password ${CATALINA_OPTS}" CATALINA_OPTS="-Dcom.sun.management.jmxremote.access.file=/atlassian-jira-software-x.y.z-standalone/jmxremote.access ${CATALINA_OPTS}"
  
  x.y.z stands for the Jira version you're using.
  
  For more details, see the Using Password Authentication section in Monitoring and Management Using JMX Technology.
  
  Additionally, to access the JMX properties, you might need to configure SSL. For more details, see the SSL sections in Monitoring and Management Using JMX Technology.
  
  JAVA_OPTS
  
  Using JAVA_OPTS works to expose the JMX MBeans for remote access but causes errors to be thrown during Jira shutdown.
  
  Windows
  
  JAVA_OPTS=-Dcom.sun.management.jmxremote %JAVA_OPTS% JAVA_OPTS=-Dcom.sun.management.jmxremote.port=8099 %JAVA_OPTS% JAVA_OPTS=-Dcom.sun.management.jmxremote.authenticate=false %JAVA_OPTS%
  
  Linux
  
  JAVA_OPTS="-Dcom.sun.management.jmxremote ${JAVA_OPTS}" JAVA_OPTS="-Dcom.sun.management.jmxremote.port=8099 ${JAVA_OPTS}" JAVA_OPTS="-Dcom.sun.management.jmxremote.authenticate=false ${JAVA_OPTS}" export JAVA_OPTS
2. Decide how you will secure your remote connection. See Remote Monitoring and Management for more information.
  Although it is possible to disable authentication, we don't recommend doing this on a production system.
3. Start JConsole. You'll find it in the bin directory of the JDK installation directory.
4. Select Remote Process.
5. Enter your hostname and port. This is the port you specified earlier, not the Jira port.
6. Select Connect.
7. After connecting, expand the com.atlassian.jira property that groups all the metrics.
See Using JConsole for more information on remote monitoring.

Known security issues

We’re providing a robust fix for a potential security vulnerability that may be caused by an RCE (remote code execution) JMX attack. During this attack, a remote user with valid credentials for JMX monitoring can execute arbitrary code on Jira Data Center via Java Deserialization, even if this user’s account is readOnly (montiorRole).

To prevent fabricated data from getting into the system through requests, we're using a blocklist deserialization filter based on ObjectInputFilter from JVM.

If you use a custom JDK and miss appropriate classes in your classpath based on the Java version, your Jira node won’t be started.

atlassian.jira.log will contain the following error: BlocklistDeserializationFilter has not been set up. It means that your Java environment has some security issues.

To eliminate the error and boost the security of your Jira instance, make sure your JDK contains the following classes:

For JDK 8: the class sun.misc.ObjectInputFilter must be enabled in the classpath.
For JDK 11 and later: the class java.io.ObjectInputFilter must be enabled in the classpath.

In-product diagnostics available through JMX

Since Jira 9.3, we've introduced a set of database connectivity metrics for in-product diagnostics available through JMX.

Learn about the updates in Jira 9.5 and later

In Jira 9.5, in-product diagnostics was complemented with more new metrics: HTTP connection metrics and mail queue metrics.

In Jira 9.8, we added a few new mail queue metrics allowing you to get a more detailed picture of mail queue contents and to collect more data for better performance monitoring.

In Jira 9.11, we introduced new infrastructure metrics for monitoring the health and performance of your instance infrastructure: outgoing and incoming mail servers, external user directories, shared and local home directories, and node communication for Data Center instances.

In-product diagnostics (IPD) provides greater insights for you and our Support into how running instances are operating.

IPD uses additional metrics handling Jira’s interactions with its database. Using database connectivity metrics, you’ll efficiently identify what in your environment or infrastructure might cause the performance issues.

The feature is disabled by default. Live metrics are available in the following formats:

as new JMX MBeans
as a history of snapshots of the JMX values in the new IPD log file atlassian-jira-ipd-monitoring.log

The log file is available in the {jira_home}\log folder where you can find all the existing log files. The log file is also included in the Support Zip file, created in the ATST plugin. If needed, you can generate the Support Zip file in the Atlassian troubleshooting & support tools plugin and send the file to Atlassian Support, where we have internal tools to interpret it. Learn more about the plugin

Communication

The feature communicates in the following ways:

JMX: JMX MBeans are updated periodically based on an internal schedule.
The log file atlassian-jira-ipd-monitoring.log: JMX values are snapshotted and recorded to the log file on a configurable schedule. By default, the JMX values are polled and written to the log file every 60 seconds. (This parameter is up to date since Jira 9.3 EAP 02.)

In-product diagnostics metrics

Expand the following sections to learn more about the metrics available for in-product diagnostics.

To use the metrics, make sure you’ve enabled JMX.

Database connectivity metrics

See the metrics provided by the IPD and their descriptions in the following table.

MBean ObjectName	Metric description
`com.atlassian.jira:type=metrics, category00=db,category01=connection, category02=failures,name=counter`	db.connection.failures.counter The count of database connection failures since the last restart
`com.atlassian.jira:type=metrics, category00=db,category01=connection, category02=latency,name=statistics`	db.connection.latency.statistics Aggregated statistics of latency since the last restart
`com.atlassian.jira:type=metrics, category00=db,category01=connection, category02=latency,name=value`	db.connection.latency.value The latest measure of latency when querying the database
`com.atlassian.jira:type=metrics, category00=db,category01=connection, category02=pool,category02=numActive, name=statistics`	db.connection.pool.numActive.statistics Aggregated statistics of the number of active connections in the database connection pool since the last restart
`com.atlassian.jira:type=metrics, category00=db,category01=connection, category02=pool,category02=numActive, name=value`	db.connection.pool.numActive.value The latest measure of the number of active connections in the database connection pool
`com.atlassian.jira:type=metrics, category00=db,category01=connection, category02=pool,category02=numIdle, name=statistics`	db.connection.pool.numIdle.statistics Aggregated statistics of the number of idle connections in the database connection pool since the last restart
`com.atlassian.jira:type=metrics, category00=db,category01=connection, category02=pool,category02=numIdle, name=value`	db.connection.pool.numIdle.value The latest measure of the number of idle connections in the database connection pool
`com.atlassian.jira:type=metrics, category00=db,category01=connection, category02=state,name=value`	db.connection.state.value The latest indicator of the state of the connection to the database

HTTP connection metrics

See the metrics provided by the IPD and their descriptions in the following table.

MBean ObjectName	Metric description
`com.atlassian.jira:type=metrics, category00=http,category01=connection, category02=pool,category03=numActive, name=value`	http.connection.pool.numActive.value The latest measure of the number of active connections in the HTTP connection pool
`com.atlassian.jira:type=metrics, category00=http,category01=connection, category02=pool,category03=numIdle, name=value`	http.connection.pool.numIdle.value The latest measure of the number of idle connections in the HTTP connection pool
`com.atlassian.jira:type=metrics, category00=http,category01=connection, category02=pool,category03=numMax, name=value`	http.connection.pool.numMax.value The maximum number of threads to be created by the connector and made available for requests
`com.atlassian.jira:type=metrics, category00=http,category01=connection, category02=sessions,category03=active, name=value`	http.connection.sessions.active.value The latest measure of the number of active user sessions
`com.atlassian.jira:type=metrics, category00=http,category01=connection, category02=sessions,category03=active, name=statistics`	http.connection.sessions.active.statistics Aggregated statistics of the number of active user sessions
`com.atlassian.jira:type=metrics, category00=http,category01=connection, category02=sessions,category03=recent, name=value`	http.connection.sessions.recent.value The latest measure of the number of recent user sessions. A recent session is the session that has been active in the one last hour.
`com.atlassian.jira:type=metrics, category00=http,category01=requests, name=value`	http.requests.value The latest measure of the total number of HTTP requests per minute
`com.atlassian.jira:type=metrics, category00=http,category01=requests, name=statistics`	http.requests.statistics Aggregated statistics of the total number of HTTP requests per minute

Mail queue metrics

See the metrics provided by the IPD and their descriptions in the following table.

MBean ObjectName	Metric description
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=numErrors,name=value`	mail.queue.numErrors.value The latest measure of the number of items in an error mail queue
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=numItems,name=value`	mail.queue.numItems.value The latest measure of the number of items in a mail queue
`com.atlassian.jira:type=metrics,` `category00=mail,category01=queue,` `category02=numErrors,name=statistics`	mail.queue.numErrors.statistics Aggregated statistics of the number of items in an error mail queue
`com.atlassian.jira:type=metrics,` `category00=mail,category01=queue,` `category02=numItems,name=statistics`	mail.queue.numItems.statistics Aggregated statistics of the number of items in a mail queue
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=numItemsAddedPerMin, name=value`	mail.queue.numItemsAddedPerMin.value The latest measure of the number of items added to a mail queue per minute
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=numErrorsAddedPerMin, name=value`	mail.queue.numErrorsAddedPerMin.value The latest measure of the number of items added to an error mail queue per minute
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=numItemsAddedPerMin, name=statistics`	mail.queue.numItemsAddedPerMin.statistics Aggregated statistics of the number of items added to a mail queue per minute
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=numErrorsAddedPerMin, name=statistics`	mail.queue.numErrorsAddedPerMin.statistics Aggregated statistics of the number of items added to an error mail queue per minute
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=numItemsProcessedPerMin, name=value`	mail.queue.numItemsProcessedPerMin.value The latest measure of the number of items processed by a mail queue per minute
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=numItemsProcessedPerMin, name=statistics`	mail.queue.numItemsProcessedPerMin.statistics Aggregated statistics of the number of items processed by a mail queue per minute
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=numEmailsSentPerMin, name=value`	mail.queue.numEmailsSentPerMin.value The latest measure of the number of emails sent by the SMTP server per minute
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=numEmailsSentPerMin, name=statistics`	mail.queue.numEmailsSentPerMin.statistics Aggregated statistics of the number of emails sent by the SMTP server per minute
`com.atlassian.jira:type=metrics, category00=mail,category01=queue, category02=jobRunning,name=value`	mail.queue.jobRunning.value The latest indicator of the state of a mail queue job 1 – the mail queue job is currently running 0 – the mail queue job is currently not running

Infrastructure metrics

See the metrics provided by the IPD and their descriptions in the following table.

MBean ObjectName	Metric description
Cluster metrics
`com.atlassian. jira:type=metrics, category00=node, category01=latency, name=statistics, tag.destNode<nodeId>`	node.latency.statistics destNode=<nodeId> Aggregated statistics of communication latency to the node (`<nodeId>`). There's a metric for all cluster nodes except for itself. These metrics are emitted on every node in the cluster. While latency to all other nodes is measured, `<nodeID>` will never be the same as the node where the metric is emitted.
`com.atlassian. jira:type=metrics, category00=node, category01=connection, category02=state, name=custom, tag.destNode=<nodeId>`	node.connection.state.custom destNode=<nodeId> The state of communication with another node (`<nodeId>`). There's a metric for all cluster nodes except for itself. These metrics are emitted on every node in the cluster. While latency to all other nodes is measured, `<nodeID>` will never be the same as the node where the metric is emitted.
Mail server metrics
`com.atlassian.jira: type=metrics, category00=mail, category01=outgoing, category02=connection, category03=state, name=custom`	mail.outgoing.connection.state.custom The state of connection to an outgoing SMTP mail server. The metric is available if the SMTP server is configured.
`com.atlassian.jira: type=metrics, category00=mail, category01=incoming, category02=connection, category03=state, name=custom,tag.serverName=<mailName>`	mail.incoming.connection.state.custom serverName=<mailName> The state of connection to an incoming mail server (`<mailName>`). There's a separate metric for each configured incoming mail server.
Shared storage metrics
`com.atlassian. jira:type=metrics, category00=home, category01=shared, category02=write, category03=latency, name=value`	home.shared.write.latency.value The median latency of writing a small file (~30 bytes) to the shared home
`com.atlassian. jira:type=metrics, category00=home, category01=shared, category02=write, category03=latency, name=statistics`	home.shared.write.latency.statistics Aggregated latency statistics of writing a small file (~30 bytes) to the shared home
Local storage metrics
`com.atlassian. jira:type=metrics, category00=home, category01=local, category02=write, category03=latency, category04=synthetic, name=value`	home.local.write.latency.synthetic.value The median latency of writing a small file (~30 bytes) to the local home with guaranteed persistence
`com.atlassian. jira:type=metrics, category00=home, category01=local, category02=write, category03=latency, category04=synthetic, name=statistics`	home.local.write.latency.synthetic.statistics Aggregated latency statistics of writing a small file (~30 bytes) to the local home with guaranteed persistence
`com.atlassian. jira:type=metrics, category00=home, category01=local, category02=write, category03=latency, category04=indexwriter, name=statistics`	home.local.write.latency.indexwriter.statistics Aggregated latency statistics to persist the current Lucene index buffer The metric is updated only when Lucene persists latest index changes.
User directory metrics
`com.atlassian. jira:type=metrics, category00=user, category01=directory, category02=connection, category03=latency, name=value, tag.userDirName=<directoryName>`	user.directory.connection.latency.value userDirName=<directoryName> The latest value of latency to search a single user in the external user directory (`<directoryName`>). There's a metric for every external user directory.
`com.atlassian. jira:type=metrics, category00=user, category01=directory, category02=connection, category03=latency, name=statistics, tag.userDirName=<directoryName>`	user.directory.connection.latency.statistics userDirName=<directoryName> Aggregated latency statistics to search a single user in the external user directory (`<directoryName>`). There's a metric for every external user directory.
`com.atlassian. jira:type=metrics, category00=user, category01=directory, category02=connection, category03=state, name=custom, tag.userDirName=<directoryName>`	user.directory.connection.state.custom userDirName=<directoryName> The state of connection to an external user directory (`<directoryName>`). Checks if a connection to a remote server can be established. There's a metric for every external user directory.

To learn more details about the infrastructure metrics, check the article Interpreting infrastructure metrics for in-product diagnostics.

To find more details on cross-product metrics, check the article Interpreting cross-product metrics for in-product diagnostics.

To check the more detailed definitions of Jira-specific metrics, check the article Interpreting Jira-specific metrics for in-product diagnostics.

Enabling In-product diagnostics monitoring

IPD monitoring is enabled by default. To manage it:

From the top navigation bar select Administration > System.
In the left-side panel, go to System Support and select Monitoring.
Use the In-product diagnostics monitoring toggle to enable or disable IPD monitoring.
The toggle is also available in version 9.4.3. Check Jira Software release notes for updates.

Enabling In-product diagnostics monitoring in the user interface

REST API

In Jira 9.5, we've introduced a new REST API endpoint for managing the IPD monitoring, specifically the In-product diagnostics monitoring toggle in the user interface: /rest/api/2/monitoring/ipd.

GET

Returns the state of the IPD functionality:

true: In-product diagnostics monitoring is enabled.
false: In-product diagnostics monitoring is disabled.

Sample response:

{
  "enabled": true
}

POST

Sets the state of IPD functionality. The response is empty and the successful result is confirmed with an HTTP 20x status response.

Sample response:

{
  "enabled": false
}

Log formatting

Writing to atlassian-jira-ipd-monitoring.log is done via log4j. Its configuration is managed in log4j.properties.

#####################################################
# In-product diagnostics monitoring logging
#####################################################

log4j.appender.ipd=com.atlassian.jira.logging.JiraHomeAppender
log4j.appender.ipd.File=atlassian-jira-ipd-monitoring.log
log4j.appender.ipd.MaxFileSize=20480KB
log4j.appender.ipd.MaxBackupIndex=5
log4j.appender.ipd.layout=com.atlassian.logging.log4j.NewLineIndentingFilteringPatternLayout
log4j.appender.ipd.layout.ConversionPattern=%d %m%n

log4j.logger.ipd-monitoring = INFO, filelog
log4j.additivity.ipd-monitoring = false
log4j.logger.ipd-monitoring-data-logger = INFO, ipd
log4j.additivity.ipd-monitoring-data-logger = false

Log contents

By default, a concise set of data is included in each log entry. An extended set of data can be logged by enabling the com.atlassian.jira.in.product.diagnostics.extended.logging feature flag.

To enable the extended data:

Go to <JIRA_URL>/secure/admin/SiteDarkFeatures!default.jspa, where <JIRA_URL> is the base URL of your Jira instance.
In the Enable dark feature text area, enter com.atlassian.jira.in.product.diagnostics.extended.logging.enabled. Select Add. Learn how to manage dark features
1. To disable the extended data, in the Site Wide Dark Features panel, find com.atlassian.jira.in.product.diagnostics.extended.logging.enabled and select Disable.

In the following tables, see the structures of the concise vs extended logging formats.

The metrics in JMX always go in the extended format.

Learn more about the metric attributes

Concise data

MBean Type

Properties

Attributes

Counter

timestamp

label

attributes

_count

Value

_value

Statistics

_99thPercentile

_max

_min

_mean

See the example of the concise log line format

2022-09-06 18:37:48,011 IPDMONITORING {"timestamp":"1662453468","label":"DB.CONNECTION.LATENCY.STATISTICS","attributes":{"_mean":"6.704470250010645E-25","_max":"1.0","_99thPercentile":"0.0","_min":"0.0"}}

Extended data

The metrics in JMX always go in the extended format.

Learn more about the metric attributes

MBean Type

Properties

Attributes

Counter

timestamp

label

attributes

objectName

_count

_fifteenMinuteRate

_fiveMinuteRate

_meanRate

_oneMinuteRate

_rateUnit

Value

_value

_number

Statistics

_50thPercentile

_75thPercentile

_95thPercentile

_98thPercentile

_99thPercentile

_999thPercentile

_count

_min

_max

_mean

_stdDev

_durationUnit

_fifteenMinuteRate

_fiveMinuteRate

_meanRate

_oneMinuteRate

_rateUnit

See the example of the extended log line format

2022-09-06 18:38:48,015 IPDMONITORING {"timestamp":"1662453528","label":
"DB.CONNECTION.LATENCY.STATISTICS","objectName":
"com.atlassian.jira:category00\u003ddb,category01\u003dconnection,category02\
u003dlatency,name\u003dstatistics,type\u003dmetrics",
"attributes":{"_oneMinuteRate":"0.02012497818617073","_50thPercentile":"0.0",
"_mean":"1.9379304604014412E-25","_max":"1.0","_stdDev":"4.40219315841711E-13",
"_98thPercentile":"0.0","_meanRate":"0.003612560785169162","_rateUnit":
"events/second","_99thPercentile":"0.0","_count":"16","_durationUnit":
"milliseconds","_75thPercentile":"0.0","_fiveMinuteRate":
"0.005912972095043379","_fifteenMinuteRate":"0.0037696657500141968",
"_999thPercentile":"0.0","_95thPercentile":"0.0","_min":"0.0"}}

Definitions of metric attributes

Expand the following sections to learn more about metric attributes.

Counter metric attributes

Attribute	Definition
`_count`	The number of occurrences of a metric within the current time window
`_fifteenMinuteRate`	The number of occurrences of a metric over the last 15 minutes
`_fiveMinuteRate`	The number of occurrences of a metric over the last five minutes
`_meanRate`	The mean rate at which events have occurred since the meter was created
`_oneMinuteRate`	The number of occurrences of a metric over the last one minute
`_rateUnit`	The unit of measure used for rates

Pay attention to the following attributes: _oneMinuteRate, _fiveMinuteRate, and _fifteenMinuteRate.

The _count gives no indication of how the measurements have changed over time. A sense of recency is provided with the minute rates.

Value metric attributes

Attribute	Definition
`_value`	A most recently sampled value of the metric
`_number`	Contains the same value as the `_value` attribute

Statistics metric attributes

The metrics of the statistics MBean type are also known as aggregated values. They provide statistics for the items that have been subjected to any changes over a period of time. For example, for the items that have been processed in a mail queue or added to an error mail queue.

Time window

Unless stated, aggregated values are calculated over a sliding time window. It covers the last five minutes, approximately.

Percentile values are calculated using a reservoir sampling technique. This technique uses a small, manageable set of values that is statistically representative of the data stream as a whole, hence reducing the quantity of data that must be held in memory.

Resets

Outside of the sliding time window, all aggregated values are reset:

After each system restart.
After each time JMX monitoring or in-product diagnostic metrics are enabled.

Learn more about JMX monitoring and in-product diagnostic in other Data Center products:

In the following table, find the definitions of statistics metric attributes.

Attribute	Definition
`_50thPercentile`	A measured value below which 50% of all measurements can be found within the current time window; also referred to as the median value. This attribute provides an alternative to the mean as a representation of the middle measurement. The median is less likely to be skewed by outlier values than the mean.
`_75thPercentile`	The measured value below 75% of all measurements that can be found within the current time window; the third quartile value
`_95thPercentile`	The measured value below 95% of all measurements that can be found within the current time window
`_98thPercentile`	The measured value below 98% of all measurements that can be found within the current time window
`_99thPercentile`	The measured value below 99% of all measurements that can be found within the current time window
`_999thPercentile`	The measured value below 999% of all measurements that can be found within the current time window
`_count`	The number of occurrences of a metric within the current time window
`_min`	The minimum measured value within the current time window
`_max`	The maximum measure value within the current time window; the statistical range between `_max` and `_min` provides a measure of values variability
`_mean`	The average value within the current time window. This attribute can be skewed by large outlier measurements. In such cases, the `_50thPercentile` provides a better measure of the middle value.
`_stdDev`	A measure of the data variability. A low standard deviation indicates that the values tend to be close to the mean of the set, while a high standard deviation indicates that the values are spread out over a wider range of values.
`_durationUnit`	The unit of measure used for durations
`_fifteenMinuteRate`	The number of occurrences of a metric over the last 15 minutes
`_fiveMinuteRate`	The number of occurrences of a metric over the last five minutes
`_meanRate`	The mean rate at which events have occurred since the meter was created
`_oneMinuteRate`	The number of occurrences of a metric over the last one minute
`_rateUnit`	The unit of measure used for rates

Pay attention to the following attributes: _oneMinuteRate, _fiveMinuteRate, and _fifteenMinuteRate.

The _count gives no indication of how the measurements have changed over time. A sense of recency is provided with the minute rates.

Processing properties

JMX logging polling interval is set to 60 seconds and can't be modified.
Log file polling interval is set to 60 seconds and can be changed by using the system property jira.diagnostics.ipdlog.poll.seconds.
By default, the JMX values are polled and written to atlassian-jira-ipd-monitoring.log.

Administering Jira applications Support

Products

Jira Software

Jira Service Management

Jira Work Management

Confluence

Bitbucket

Resources

Documentation

Community

Suggestions and bugs

Marketplace

Billing and licensing

Live monitoring using the JMX interface

Monitor application performance

On this page

Still need help?

What is JMX?

Metrics collected by Jira

Metrics collected by Assets in Jira Service Management

Monitoring Jira

Enabling JMX monitoring in Jira

Monitoring with JConsole

Known security issues

In-product diagnostics available through JMX

Communication

In-product diagnostics metrics

Enabling In-product diagnostics monitoring

REST API

Log formatting

Log contents

Concise data

Extended data

Definitions of metric attributes

Processing properties

Page

Viewport

Confluence

Versions

Live monitoring using the JMX interface

Monitor application performance

On this page

Related content

Still need help?

What is JMX?

Metrics collected by Jira

Metrics collected by Assets in Jira Service Management

Monitoring Jira

Enabling JMX monitoring in Jira

Monitoring with JConsole

Known security issues

In-product diagnostics available through JMX

Communication

In-product diagnostics metrics

Enabling In-product diagnostics monitoring

REST API

Log formatting

Log contents

Concise data

Extended data

Definitions of metric attributes

Processing properties

Related content