Indexing fails due to immense field
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Problem
Both background and foreground reindex requests fail due to a field that has more than 32766 bytes in the field.
The following appears in the atlassian-jira.log
2019-07-02 16:32:24,247 IssueIndexer:thread-9 WARN 10.x.x.x /secure/admin/jira/IndexReIndex!reindex.jspa [c.a.jira.index.AccumulatingResultBuilder] Document contains at least one immense term in field="<fieldname>" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[list of entries in the field]...', original message: bytes can be at most 32766 in length; got 32767
2019-07-02 16:32:24,248 JiraTaskExectionThread-2 WARN 10.x.x.x /secure/admin/jira/IndexReIndex!reindex.jspa [c.a.jira.index.AccumulatingResultBuilder] Indexing failed for Issue - '<#####>'
Diagnosis
Environment
This affects JIRA 8.0 and above.
- In JIRA 8.0, Lucene was upgraded from 3.3 to 7.3. Lucene will now throw an exception instead of silently failing.
- See Lucene upgrade for more information.
Cause
Lucene limits the length of indexed fields to 32766 bytes. Fields that are continually incremented (such as SLA, individual permissions, or customfields) have a higher likelihood of exceeding the length limit. The application and the database allow these to increment and do not limit the field length, but Lucene cannot parse those immense fields that exceed the byte length, so it fails to index that field.
In older versions of Lucene, it would silent fail, but after LUCENE-5472 this will throw an exception.
Workaround
There is no workaround at this time.
There is an open bug report: - JSWSERVER-20133Getting issue details... STATUS
If you are affected by this issue, please go to the bug and click on "This affects my team."
Resolution
In order to allow indexing to run, you must either truncate, disable, or delete the problematic field.
If the field is managed by a plugin, you can disable that plugin.
Limit the input points. For example, if the error indicates a permission field, limit the individuals or groups that are included in the permission field.
If it is a customfield that is no longer needed, remove it from active screens then trim it in the database.