Full GC event keep occurring during the last stage of Assets import
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
The last stage of Assets import is when object references get updated. In some scenarios, Jira performance could get impacted during this stage, for example, Jira stops responding or responds slowly when an Assets import is at the last stage but resumes to its usual state as soon as the import is done.
Environment
Any supported version of Jira with Assets in use
Diagnosis
- Frequent full GC events can be seen from GC log. To determine full GC, refer to How to define Xmx based on GC logs: How to read the graphs below. The individual GC pauses are displayed with black vertical bars, for example, when we load the gc logs into GC viewer:
In insight_import.log, in the same time range when GC kept happening (based on the screenshot above, on November 11 between 01:10 to about 02:10), we could see references being removed from many objects due to cardinality constraint.
2022-11-11 01:10:45,793 [insight-InsightImportThreadGroup-worker-thread-4] | Removed references from object NVD-287445 due to cardinality constraint on attribute ObjectTypeAttributeBean [id=1265, name=Source, type=REFERENCED_OBJECT, defaultType=NONE] original values [[null(94513)], [null(287443)], [null(47951)]] to values [[null(94513)]] 2022-11-11 01:10:46,634 [insight-InsightImportThreadGroup-worker-thread-5] | Removed references from object NVD-287448 due to cardinality constraint on attribute ObjectTypeAttributeBean [id=1265, name=Source, type=REFERENCED_OBJECT, defaultType=NONE] original values [[null(94513)], [null(287443)], [null(47951)]] to values [[null(94513)]] 2022-11-11 01:10:46,684 [insight-InsightImportThreadGroup-worker-thread-1] | Removed references from object NVD-287444 due to cardinality constraint on attribute ObjectTypeAttributeBean [id=1265, name=Source, type=REFERENCED_OBJECT, defaultType=NONE] original values [[null(94513)], [null(287443)], [null(47951)]] to values [[null(94513)]] ... 2022-11-11 02:12:02,119 [insight-InsightImportThreadGroup-worker-thread-6] | Removed references from object NVD-209589 due to cardinality constraint on attribute ObjectTypeAttributeBean [id=1296, name=Product, type=REFERENCED_OBJECT, defaultType=NONE] original values [[null(30359)], [null(77102)]] to values [[null(30359)]] 2022-11-11 02:12:23,029 [insight-InsightImportThreadGroup-worker-thread-1] | Removed references from object NVD-209591 due to cardinality constraint on attribute ObjectTypeAttributeBean [id=1296, name=Product, type=REFERENCED_OBJECT, defaultType=NONE] original values [[null(25905)], [null(72780)]] to values [[null(25905)]] [[null(1682969)]]
Cause
In this scenario, the application has just enough resources for the usual processes to run. When the Assets import required more resources to remove references from existing objects due to cardinality constraint, full GC events were triggered to reclaim the space
Solution
If the affected attribute should not have the cardinality constraint, change the settings accordingly.
- Determine the affected attribute(s) from the log. Using the example log shared above, we could see that the affected attributes are Source with the attribute ID 1265, and Product with the attribute ID 1296.
- Edit the configuration of the affected attributes. Set a higher number of maximum value of the cardinality or select Unlimited to allow as many values as possible.
- Run another import or wait for the next schedule to start.
If the cardinality constraint is expected, this might mean that Jira needs more heap memory assigned or there's other factor contributing to frequent full GC at the same time when Assets import happens. Please contact Atlassian Support for the investigation.