Full reindex fails with message Duplicate keys mapped in Jira
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
When performing a full reindex in Jira, it fails with several WARN messages about "Duplicate keys mapped" like this below:
1
2021-03-12 15:59:56,965-0300 IssueIndexer:thread-1 WARN admin 959x1839x1 1wtf9jg 0:0:0:0:0:0:0:1 /secure/admin/jira/IndexReIndex!reindex.jspa [c.a.jira.index.AccumulatingResultBuilder] java.lang.IllegalStateException: Duplicate keys mapped to com.atlassian.jira.issue.customfields.vdi.CustomFieldPrefetchedData@6934d589 / com.atlassian.jira.issue.customfields.vdi.CustomFieldPrefetchedData@5c73dbed for custom field id customfield_10221And later the reindex failure message:
1
2021-03-12 15:59:57,136-0300 JiraTaskExecutionThread-3 ERROR admin 959x1839x1 1wtf9jg 0:0:0:0:0:0:0:1 /secure/admin/jira/IndexReIndex!reindex.jspa [c.a.j.util.index.CompositeIndexLifecycleManager] Reindex All FAILED. ...Environment
All versions of Jira 8.
Diagnosis
1) Look for the reindex all failure in the logs:
1
cat atlassian-jira.log | grep -i "reindex all failed"2) Look for "Duplicate keys mapped" messages and their respective customfields:
1
cat atlassian-jira.log | grep "Duplicate keys mapped to" | egrep -o "customfield_.*" | sort | uniq -cThis should output each customfield and the number of messages for each:
Example output
1
2
3025 customfield_10250
150 customfield_102513) Check if there are any fields that have values in both the label and customfieldvalue tables:
1
2
3
4
5
6
select distinct f.id, f.cfname
from label l
join customfieldvalue v on v.customfield = l.fieldid
join customfield f on f.id = v.customfield
where l.fieldid is not null and v.stringvalue is not null
and f.customfieldtypekey = 'com.atlassian.jira.plugin.system.customfieldtypes:labels';Confirm their ids are the same that showed up on the logs (in step #2).
Cause
There's a label-type customfield with values stored in the customfieldvalue table instead of the label table only.
This scenario shouldn't exist and Jira's reindex doesn't completely succeeds, leaving the index incomplete or in a non-reliable state (eg. filter results show different values than the issue screen).
Root cause
We don't know yet how label-type fields could have values stored in the customfieldvalue table. Maybe it's the work of 3rd party apps or custom scripts running in Jira (through Script-enabling apps) — or a direct database insert.
Solution
We should remove the invalid entries from the customfieldvalue for those label-type fields table and perform a full reindex again:
1
delete from customfieldvalue where customfield in (10250, 10251); -- example IdsAlways back up your data before making any database modifications. If possible, test any alter, insert, update, or delete SQL commands on a staging server first.
Was this helpful?