Configuring OpenSearch for Confluence
As Confluence instances grow in size and scale, the default search engine, Lucene, may be slower to index and return search results. To address this, Confluence Data Center now offers an alternative search engine as an opt-in feature — OpenSearch. OpenSearch can take advantage of multi-node instances to manage process-intensive indexing. In the user interface, the search experience remains the same for users.
On this page:
Recommendations
- Confluence 9.0 or above
- OpenSearch 2.11 or above
Set up OpenSearch
Setting up OpenSearch requires three actions:
Provision an OpenSearch cluster.
Configure Confluence to use OpenSearch.
Migrate your Lucene index.
All these actions are covered on this page.
Provision an OpenSearch cluster
Confluence will automatically create the required indexes when you provision an OpenSearch cluster.
Provision an OpenSearch cluster, either on premises or as a managed service (for example, AWS OpenSearch Service).
For detailed instructions, refer to the OpenSearch documentation for setting up a cluster.Configure your cluster with the appropriate level of security.
Refer to Securing an OpenSearch cluster for the recommended security configuration.
Configure Confluence to use OpenSearch
To configure Confluence to use OpenSearch:
tell Confluence that you’re using OpenSearch
provide Confluence with the OpenSearch cluster location and authentication details
You can do this by setting properties in either the confluence.cfg.xml
file or via system properties. Note that if both are present, system properties will take precedence.
These minimum configuration options are covered in the tables below, and other configuration options are available too. For the full list of system properties, refer to Refer to Recognized System Properties.
Tell Confluence to use OpenSearch
Set the search.platform
property to opensearch
(the default is lucene
).
Example
In
confluence.cfg.xml
:<property name="search.platform">opensearch</property>
Alternatively, as a system property:
-Dsearch.platform=opensearch
Provide cluster location and authentication details
Use either basic auth (username/password) or AWS Identity and Access Management (IAM) to authenticate with your OpenSearch cluster.
Option 1: authenticate with basic authentication
Property
opensearch.http.url (This is t
he HTTP(S) URL of the OpenSearch cluster, or multiple URLs separated by commas.)opensearch.username
opensearch.password
Example
- In
confluence.cfg.xml
:
<property name="opensearch.http.url">https://10.1.1.10:9200,https://10.1.1.11:9200</property>
<property name="opensearch.username">admin</property>
<property name="opensearch.password">my-password</property>
- Alternatively, as system properties:
-Dopensearch.http.url=https://10.1.1.10:9200,https://10.1.1.11:9200 \
-Dopensearch.username=admin \
-Dopensearch.password=my-password
Option 2: authenticate to Amazon OpenSearch Service with AWS request signing
AWS request signing allows you to use Amazon OpenSearch Service with Confluence. This lets you secure your Amazon OpenSearch Service cluster to only allow requests from the IAM user that the node Confluence is running on inside of AWS EC2 has. To use Amazon OpenSearch Service with request signing, you must specify the following properties.
Property
opensearch.aws.host
(the URL of the OpenSearch domain)opensearch.aws.region
(the AWS region of the OpenSearch domain)
Example
- In
confluence.cfg.xml
:
<property name="opensearch.aws.host">vpc-confluence.us-east-1.es.amazonaws.com</property>
<property name="opensearch.aws.region">us-east-1</property>
- Alternatively, as system properties:
-Dopensearch.aws.host=vpc-confluence.us-east-1.es.amazonaws.com \
-Dopensearch.aws.region=us-east-1
Configure multiple Confluence instances
You can configure multiple Confluence instances to use the same OpenSearch instance. If you choose to do this, configure each instance with different index names to avoid clashing indexes. You can change the index names for an instance in two ways:
via the system properties
via the
confluence.cfg.xml
file
If they’re set differently in both, Confluence will use the system property value.
You will need to configure an index name in four properties:
opensearch.index.content.name
opensearch.index.change.name
opensearch.index.custom.prefix
opensearch.index.journal.name
For example, they might look like this in your system properties:
-Dopensearch.index.content.name=confluence-instance1-content -Dopensearch.index.change.name=confluence-instance1-change -Dopensearch.index.custom.prefix=confluence-instance1-custom- -Dopensearch.index.journal.name=confluence-instance1-journal
Check your configuration
You can verify that Confluence is configured with OpenSearch on the System Information page. Check that the Search platform displays “OpenSearch”. You can also view the Instance Health page to verify that OpenSearch is functioning properly.
Migrate the Lucene index
Now that you’ve reconfigured Confluence to use OpenSearch, it’s time to repopulate the index with the existing data.
Simple index migration with downtime
The simplest approach is to rebuild your index as soon as you’ve reconfigured your instance to use OpenSearch. However, this will cause your search functionality to be temporarily unavailable, or returning incomplete results, until the index is fully rebuilt. This is the only approach available for instances on a single node. You can use the Content Indexing admin UI to rebuild your index.
If your instance runs on a clustered configuration, you can avoid any search downtime by following the steps below.
Index migration without downtime
If your Confluence instance is running in a clustered configuration, you can maintain search availability during this migration process. The steps below describe this process. Make sure you’re famliar with privisioning dark nodes, which are Confluence nodes that don’t serve the regular user traffic from the load balancer.
From your Confluence cluster running Lucene, provision a dark node.
Configure the dark node with OpenSearch so it’s connected to the target OpenSearch cluster.
Log in to the dark node and verify on the System Information page that it’s running on OpenSearch.
Start the index rebuild process on the Content Indexing admin UI.
This will repopulate the OpenSearch index with the existing data, and can take some time.Once the OpenSearch index has been completely rebuilt, verify that you can search for your documents as expected on this node.
Configure the remaining cluster nodes from Lucene to OpenSearch.
Deprovision the dark node, if needed.
Notable changes
There are some key differences when switching from Lucene to OpenSearch to be aware of, as it might require some changes in your existing processes.
Production backup strategies
Currently, you might take backups of your Lucene index in the home directory, which will avoid the need for a full reindex when restoring. After switching to OpenSearch, you will need to employ a different strategy using snapshots. Refer to Production backup strategy for more details.
Search window limit
OpenSearch has a default result window limit of 10,000. This is the maximum number of results you can search for, and the default is set to limit memory usage. If your search request exceeds this limit (for example, if an existing app is programmed to search for more than that limit), you will get the following error:
Result window is too large, from + size must be less than or equal to: [10000] but was [10010]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.
If this limitation is causing a problem on your environment, contact Support or your app developers: there are memory-efficient ways to work with a large number of search results that developers can employ to avoid this limitation.
Alternatively, as an immediate workaround, you can increase the index.max_result_window
setting on your OpenSearch index. Search requests take heap memory and time proportional to the result window, so make sure your OpenSearch data node has enough memory. Consult OpenSearch documentation about this index setting.
.