The mapping for type in Elasticsearch is invalid

Still need help?

The Atlassian Community is here for you.

Ask the community

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.

Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.

*Except Fisheye and Crucible

Summary

When Bitbucket Server uses a remote Elasticsearch instance rather than the bundled version, it's possible that the remote instance has been set up incorrectly and search fails to work. This may happen if not all the steps provided on the Install and configure a remote Elasticsearch server have been performed.

The following appears in the atlassian-bitbucket.log

DEBUG [Caesium-1-4]  c.a.b.s.i.i.IndexingSynchronizationService Attempting to validate mapping in Elasticsearch
ERROR [Caesium-1-4]  c.a.b.s.i.i.IndexingSynchronizationService The mapping for type 'bitbucket-search/repository' in Elasticsearch is invalid. Errors: [Field 'quickSearchProjectName' has no value for 'search_analyzer'. Expected: "standard", Field 'quickSearchRepositoryName' has no value for 'search_analyzer'. Expected: "standard"]
ERROR [Caesium-1-4]  c.a.b.s.i.i.IndexingSynchronizationService The mapping for type 'bitbucket-search/file' in Elasticsearch is invalid. Errors: [Field 'content' has no value for 'search_analyzer'. Expected: "analyzer_code_search"]


ERROR [Caesium-1-3]  c.a.b.i.s.i.IndexingSynchronizationService The mapping for type 'bitbucket-search/file' in Elasticsearch is invalid. Errors: [Field 'path' has no value for 'analyzer'. Expected: "analyzer_path", Field 'extension' has unexpected value for 'type'. Expected: "keyword", actual: "text", Field 'filename' has no value for 'analyzer'. Expected: "analyzer_filename", Field 'content' has no value for 'analyzer'. Expected: "analyzer_code", Field 'content' has no value for 'search_analyzer'. Expected: "analyzer_code_search"]
ERROR [Caesium-1-3]  c.a.b.i.s.i.IndexingSynchronizationService The mapping for type 'bitbucket-index-state/indexstate' in Elasticsearch is invalid. Errors: [Field 'retries' has unexpected value for 'type'. Expected: "integer", actual: "long"]
ERROR [Caesium-1-3]  c.a.b.i.s.i.IndexingSynchronizationService The mapping for type 'bitbucket-project/project' in Elasticsearch is invalid. Errors: [Field 'avatarData' is missing, Field 'avatarType' is missing, Field 'key' has no value for 'analyzer'. Expected: "analyzer_keyword_lowercase", Field 'key' has no value for 'index_options'. Expected: "docs", Field 'key' has no value for 'norms'. Expected: false]
ERROR [Caesium-1-3]  c.a.b.i.s.i.IndexingSynchronizationService Mapping in Elasticsearch is present but not compatible with this version of indexing. Unable to continue, operator intervention is required.

Note: the first line will only be added if the debug log level has been enabled.

Environment

Bitbucket Server or Data Center with a remote standalone Elasticsearch instance

Cause

This is due to Elasticsearch being sent documents before it knows about the index types. The correct thing that Elasticsearch should do is to refuse to add documents to an index that it knows nothing about.

Solution

To resolve this you'll need to do one of two things.

Option #1 - preferred

  • Start a fresh Elasticsearch instance, with the configuration set up as per the documentation.
  • In case of Elasticsearch service create a new domain.

Option #2

If you are unable to create a fresh instance and would prefer to continue to use the one already created, please follow these steps

  1. Stop Elasticsearch
  2. Add the configuration lines as per the Install and configure a remote Elasticsearch server, under the "Configure Elasticsearch" heading
  3. Start Elasticsearch
  4. Retrieve the buckler username and password. They should have set this up already, when dealing with a remote instance of Elasticsearch, Bitbucket does not set up a password itself
  5. Send a DELETE request to the following URL, deleting the whole index:

    curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<ElasticsearchURL>:<Port, default 9200>/bitbucket-search/
  6. Sometimes the broken mapping is not inside the search index and it might be necessary to delete another index as well. Below are the commands to delete all possible indexes created by Bitbucket:

    curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<Elasticsearch-URL>/bitbucket-search
    curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<Elasticsearch-URL>/bitbucket-index-state
    curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<Elasticsearch-URL>/bitbucket-index-version
    curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<Elasticsearch-URL>/bitbucket-project
    curl -X DELETE -u <ElasticUsername>:<ElasticPassword> 'http://<Elasticsearch-URL>/bitbucket-repository
  7. This will cause the index to be removed. To terminate any existing connections from Bitbucket Server or Data Center, you can either

    1. Restart Bitbucket Server or
    2. Send a POST request using the following command:

      curl -v -u <username> -X POST -H 'Content-Type: application/json' -k -H 'Accept: application/json' <baseURL>/rest/indexing/latest/sync
      

      where <username> is the username of a Bitbucket Server administrator and <BaseURL> is the base URL of the Bitbucket Server or Data Center instance. This should kick off the startup checks for Bitbucket Search. 

  8. Check the logs and ensure that since the last CURL command has been sent, there are no similar errors as per above, and the search is working as expected.

  9. If deleting the whole index using the curl command does not work, an alternative is to delete the index in the filesystem.  Note that this will require some downtime.

    For Bitbucket Server using bundled Search server:

    1. Stop Bitbucket Server
    2. Create a backup of <Bitbucket-home>/shared/search/data/nodes directory
    3. Delete the contents of <Bitbucket-home>/shared/search/data/nodes directory
    4. Start Bitbucket Server

    For Bitbucket Server and Data Center using an external Search server instance:

    1. Stop Bitbucket Server
    2. Stop the Search server
    3. Create a backup of <SEARCH-SERVER_HOME>/data/local/nodes directory in your Search server
    4. Delete the contents of <SEARCH-SERVER_HOME>/data/local/nodes directory
    5. Start the Search server
    6. Start Bitbucket Server




Last modified on Nov 7, 2023

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.