Changes to index management on the Jira startup in version 9.1
Platform Notice: Data Center - This article applies to Atlassian products on the Data Center platform.
Note that this knowledge base article was created for the Data Center version of the product. Data Center knowledge base articles for non-Data Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Platform Notice: Data Center - This article applies to Atlassian products on the Data Center platform.
Note that this knowledge base article was created for the Data Center version of the product. Data Center knowledge base articles for non-Data Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Before Jira 9.1
During start-up a Jira node needs to obtain the index.
If local index exists, Jira checks how far behind the DB it is:
- If it is less than 10%, Jira starts and assumes the node replication thread will catch up asynchronously. This could cause some issues:
- users are allowed to access node (
/status
endpoint returnsRUNNING
) even when the index is not up to date; - the re-indexing triggered by the node replication thread is slow (single thread re-indexing with conditional updates), so in case of large instances the <10% index catch up could still take a long time;
- users are allowed to access node (
- If this is more than 10%, Jira requests any other node in the cluster to create an index. The start process continues and the node asynchronously waits for the index to be created. This could cause a number of issues described in
JRASERVER-72125
-
Getting issue details...
STATUS
and / or
JRASERVER-66635
-
Getting issue details...
STATUS
.
Some examples:- there is only one active node in the cluster, so there is no other node to deliver the index
- there is more than one node starting concurrently and starting nodes exchange empty or incorrect indexes
- node preparing the index failed to:
- create the index snapshot
- send the index snapshot to shared-home
- inform the waiting node that the snapshot was created
Before Jira 9.1 it was admin's responsibility to ensure that only one node starts at a time and that all nodes in the cluster are healthy, so that each of them can provide a healthy index snapshot.
New in Jira 9.1
Starting from Jira 9.1 the synchronous node start-up is enforced by the application.
Index start-up procedure ensures the local index is healthy before Jira can continue start-up. Jira will not start without healthy index.
The start-up procedure is performed under a cluster lock, which guarantees that only one node at a time executes it.
The index start-up procedure:
Re-index missing data if a local issue index is less than 10% behind the database. Since 9.3 the % is configurable.
- Load a recent index snapshot from shared-home directory if one is available. Since 9.3, the delta reindex in the snapshot recovery process is multi-threaded.
Otherwise trigger a full re-index.
- Make sure there is a fresh index snapshot on shared home (for other starting nodes)
Learn more about troubleshooting: Failed getting index on start