Migrating from Lucene to Search API
Lucene agnostic API
Before Jira 11, many Jira APIs exposed Lucene-specific data such as queries, documents, fields, and sorters. It was also possible to directly reference the Lucene index and perform any read or write operations supported by Lucene.
Starting with Jira 11, access to the Lucene index has been removed, and Search API equivalents have been introduced. These include Query, Document, Field, and FieldSort in the com.atlassian.jira.search package.
Jira 11 provides the following entry points for searching the Issues index:
com.atlassian.jira.bc.issue.search.SearchService: Provides domain layer access, such as retrieving Jira issues using a JQL query. This should be suitable for most cases.com.atlassian.jira.search.issue.IssueDocumentSearchService: Provides search document layer access, such as retrieving documents using a query. This is appropriate for more advanced operations, particularly where performance and scalability are key. Examples include:- Retrieving a limited amount of data from each issue
- Aggregating issue data
- Streaming large result sets
com.atlassian.jira.search.index.IndexSearcher: Provides the lowest level of access to the index, providing advanced operations such as direct access to issue change history or comments documents.
For detailed guidance on changes to the Jira API and Search API equivalents, explore the Jira API and Search API changes.
External data storage
When using Lucene, the index is stored on the disk of the machine where Jira is running. OpenSearch, however, is entirely external to Jira, which results in different performance characteristics, particularly concerning data transfer between Jira and the search index. Application code that compiles with the Search API may perform well with Lucene but not with OpenSearch, and vice versa.
Below you can find examples of access patterns that should be analyzed when migrating from Lucene to the Search API to ensure the best user experience across all search platforms.
Data retrieval
When Jira loads data from Lucene on disk, search requests in OpenSearch require a network call. We recommend to optimize both the volume and content of search requests using techniques such as:
- Pagination: Reduce the number of issues that need to be retrieved in a single user interaction.
- Aggregation: Minimize the need to retrieve issues into Jira by offloading processing to the Search Platform. For more details, see Migrating Lucene collectors to Search API.
- Tighten search filters: Where possible, optimize queries to return only the necessary data rather than relying on filtering within Jira.
- Specify fields to load: Reduce the amount of data retrieved per document.
Sorting
Sorting is most efficient when the indexed value can be naturally sorted, regardless of the search platform.
Before Jira 11, Custom Fields could use Lucene's native mechanisms, such as org.apache.lucene.search.SortField and FieldComparatorSource, to sort data external to the search index. The Search API provides similar functionality, but performance overheads differ between OpenSearch and Lucene. For more information, see Custom field sorting.
Lucene Collectors
Lucene Collectors allow you to execute code for each document matching a search query, which can be costly when documents are stored externally. For detailed guidance on migrating collectors to the Search API, see Migrating Lucene collectors to Search API.
Strongly typed Index Schema
Before Jira 11, Field Indexers (implementations of com.atlassian.jira.issue.index.indexers.FieldIndexer) could add any field to a Lucene Document.
The Search API Field Indexers (implementations of com.atlassian.jira.search.issue.index.indexers.FieldIndexer) are designed to be platform-agnostic. To ensure compatibility across multiple platforms, the following field definition restrictions apply:
- Fields must be pre-defined and registered as part of the index schema.
- Field names must be unique.
For detailed guidance on migrating Field Indexers to the Search API, see the migration guide.