The mapping for type in Elasticsearch is invalid
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
When Bitbucket Server uses a remote Elasticsearch instance rather than the bundled version, it's possible that the remote instance has been set up incorrectly and search fails to work. This may happen if not all the steps provided on the Install and configure a remote Elasticsearch server have been performed.
The following appears in the atlassian-bitbucket
.log
DEBUG [Caesium-1-4] c.a.b.s.i.i.IndexingSynchronizationService Attempting to validate mapping in Elasticsearch
ERROR [Caesium-1-4] c.a.b.s.i.i.IndexingSynchronizationService The mapping for type 'bitbucket-search/repository' in Elasticsearch is invalid. Errors: [Field 'quickSearchProjectName' has no value for 'search_analyzer'. Expected: "standard", Field 'quickSearchRepositoryName' has no value for 'search_analyzer'. Expected: "standard"]
ERROR [Caesium-1-4] c.a.b.s.i.i.IndexingSynchronizationService The mapping for type 'bitbucket-search/file' in Elasticsearch is invalid. Errors: [Field 'content' has no value for 'search_analyzer'. Expected: "analyzer_code_search"]
ERROR [Caesium-1-3] c.a.b.i.s.i.IndexingSynchronizationService The mapping for type 'bitbucket-search/file' in Elasticsearch is invalid. Errors: [Field 'path' has no value for 'analyzer'. Expected: "analyzer_path", Field 'extension' has unexpected value for 'type'. Expected: "keyword", actual: "text", Field 'filename' has no value for 'analyzer'. Expected: "analyzer_filename", Field 'content' has no value for 'analyzer'. Expected: "analyzer_code", Field 'content' has no value for 'search_analyzer'. Expected: "analyzer_code_search"]
ERROR [Caesium-1-3] c.a.b.i.s.i.IndexingSynchronizationService The mapping for type 'bitbucket-index-state/indexstate' in Elasticsearch is invalid. Errors: [Field 'retries' has unexpected value for 'type'. Expected: "integer", actual: "long"]
ERROR [Caesium-1-3] c.a.b.i.s.i.IndexingSynchronizationService The mapping for type 'bitbucket-project/project' in Elasticsearch is invalid. Errors: [Field 'avatarData' is missing, Field 'avatarType' is missing, Field 'key' has no value for 'analyzer'. Expected: "analyzer_keyword_lowercase", Field 'key' has no value for 'index_options'. Expected: "docs", Field 'key' has no value for 'norms'. Expected: false]
ERROR [Caesium-1-3] c.a.b.i.s.i.IndexingSynchronizationService Mapping in Elasticsearch is present but not compatible with this version of indexing. Unable to continue, operator intervention is required.
Note: the first line will only be added if the debug log level has been enabled.
Environment
Bitbucket Server or Data Center with a remote standalone Elasticsearch instance
Cause
This is due to Elasticsearch being sent documents before it knows about the index types. The correct thing that Elasticsearch should do is to refuse to add documents to an index that it knows nothing about.
Solution
To resolve this you'll need to do one of two things.
Option #1 - preferred
- Start a fresh Elasticsearch instance, with the configuration set up as per the documentation.
- In case of Elasticsearch service create a new domain.
Option #2
If you are unable to create a fresh instance and would prefer to continue to use the one already created, please follow these steps
- Stop Elasticsearch
- Add the configuration lines as per the Install and configure a remote Elasticsearch server, under the "Configure Elasticsearch" heading
- Start Elasticsearch
- Retrieve the buckler username and password. They should have set this up already, when dealing with a remote instance of Elasticsearch, Bitbucket does not set up a password itself
Send a
DELETE
request to the following URL, deleting the whole index:curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<ElasticsearchURL>:<Port, default 9200>/bitbucket-search/
Sometimes the broken mapping is not inside the search index and it might be necessary to delete another index as well. Below are the commands to delete all possible indexes created by Bitbucket:
curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<Elasticsearch-URL>/bitbucket-search curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<Elasticsearch-URL>/bitbucket-index-state curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<Elasticsearch-URL>/bitbucket-index-version curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<Elasticsearch-URL>/bitbucket-project curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<Elasticsearch-URL>/bitbucket-repository
This will cause the index to be removed. To terminate any existing connections from Bitbucket Server or Data Center, you can either
- Restart Bitbucket Server or
Send a
POST
request using the following command:curl -v -u <username> -X POST -H 'Content-Type: application/json' -k -H 'Accept: application/json' <baseURL>/rest/indexing/latest/sync
where
<username>
is the username of a Bitbucket Server administrator and<BaseURL>
is the base URL of the Bitbucket Server or Data Center instance. This should kick off the startup checks for Bitbucket Search.
Check the logs and ensure that since the last CURL command has been sent, there are no similar errors as per above, and the search is working as expected.
- If deleting the whole index using the curl command does not work, an alternative is to delete the index in the filesystem. Note that this will require some downtime.
For Bitbucket Server using bundled Search server:
- Stop Bitbucket Server
- Create a backup of
<Bitbucket-home>/shared/search/data/nodes
directory - Delete the contents of
<Bitbucket-home>/shared/search/data/nodes
directory - Start Bitbucket Server
For Bitbucket Server and Data Center using an external Search server instance:
- Stop Bitbucket Server
- Stop the Search server
- Create a backup of <
SEARCH-SERVER_HOME>/data/local/nodes
directory in your Search server - Delete the contents of <
SEARCH-SERVER_HOME>/data/local/nodes
directory - Start the Search server
- Start Bitbucket Server