How to reindex Jira Data Center without downtime
Platform Notice: Data Center Only - This article only applies to Atlassian products on the data center platform.
Summary
This article instructs how to perform a full re-index on JIRA Data Center such that users are not impacted by the downtime of one node (whilst the indexing operation is completed).
Environment
All versions of Jira Core and 8.x.
All versions of Jira Core 7.x higher than 7.1.
Solution
After end of full reindex process, The index will be propagated to the other nodes provided everything is OK with the shared home configuration and the instances are able to communicate with each other.
Whilst the foreground index node is in progress, if other nodes make indexing changes they will keep track of them in the database. Once the foreground indexing node has finished, the additional index changes will be replayed and integrity of the indexes is maintained across the cluster.
a) Manually taking nodes off the LB
- Ensure the node has a 'back-door' connector in it. This is an alternate HTTP connector that is not connected to the load-balancer / proxy, as detailed in Integrating JIRA with Apache in the Configure Tomcat section.
- Remove the node from the load-balancer configuration, so clients are still not being redirected to it.
- Access the node using the 'back-door' connector, and perform a 'lock JIRA and re-build index' as per Search Indexing.
- When the index has complete, access the node and verify everything is OK.
- If so, add the node back to the load-balancer configuration.
b) Automatic if load balancer's configured to monitor /status
Applicable if you are using JIRA 7.1+ and your LB is configured to use the /status endpoint for health-check.
- Connect to any node and run 'lock JIRA and re-build index' as per Search Indexing.
- Node health-check status will be changes to '{"state":"MAINTENANCE"}' and node will be removed from LB.
- There currently a bug in JIRA 7.6+ where the Status Code 500/503 will be returned instead, see JRASERVER-67138 - Getting issue details... STATUS
Due to JRA-59721 - Getting issue details... STATUS please update to JIRA 6.4.13 or later before performing a no downtime re-index.