Indexing fails due to immense field

Still need help?

The Atlassian Community is here for you.

Ask the community



Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible


Problem

Both background and foreground reindex requests fail due to a field that has more than 32766 bytes in the field. 

The following appears in the atlassian-jira.log

2019-07-02 16:32:24,247 IssueIndexer:thread-9 WARN 10.x.x.x /secure/admin/jira/IndexReIndex!reindex.jspa [c.a.jira.index.AccumulatingResultBuilder] Document contains at least one immense term in field="<fieldname>" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[list of entries in the field]...', original message: bytes can be at most 32766 in length; got 32767
2019-07-02 16:32:24,248 JiraTaskExectionThread-2 WARN 10.x.x.x /secure/admin/jira/IndexReIndex!reindex.jspa [c.a.jira.index.AccumulatingResultBuilder] Indexing failed for Issue - '<#####>'

Diagnosis

Environment

  • This affects JIRA 8.0 and above.

  • In JIRA 8.0, Lucene was upgraded from 3.3 to 7.3.  Lucene will now throw an exception instead of silently failing. 
  • See Lucene upgrade for more information.

Cause

Lucene limits the length of indexed fields to 32766 bytes.  Fields that are continually incremented (such as SLA, individual permissions, or customfields) have a higher likelihood of exceeding the length limit.  The application and the database allow these to increment and do not limit the field length, but Lucene cannot parse those immense fields that exceed the byte length, so it fails to index that field.  

In older versions of Lucene, it would silent fail, but after LUCENE-5472 this will throw an exception.  

Workaround

There is no workaround at this time. 

There is an open bug report:  JSWSERVER-20133 - Getting issue details... STATUS

If you are affected by this issue, please go to the bug and click on "This affects my team."

Resolution

In order to allow indexing to run, you must either truncate, disable, or delete the problematic field.  

  • If the field is managed by a plugin, you can disable that plugin.

  • Limit the input points.  For example, if the error indicates a permission field, limit the individuals or groups that are included in the permission field.  

  • If it is a customfield that is no longer needed, remove it from active screens then trim it in the database.  


DescriptionIndexing fails due to immense field
ProductJIRA Cloud, JIRA Data Center, JIRA Core, JIRA Service Desk, JIRA Software, Confluence



Last modified on Jul 5, 2019

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.