Install and configure a remote OpenSearch server

This page describes how to provision a remote OpenSearch server to work with Bitbucket Data Center.

Bitbucket Data Center requires a remote search server, as no search server is bundled or installed for Bitbucket Data Center. OpenSearch is a supported search server distribution for Bitbucket Data Center.

Bitbucket Data Center can have only one remote connection to OpenSearch for your cluster. This may be a standalone OpenSearch installation or a clustered installation behind a load balancer. 

For details on how Bitbucket uses the search server, including troubleshooting tips and frequently asked questions, see Administer code search.

Step 1: Install OpenSearch on a remote machine

We don't provide specific instructions for installing OpenSearch, but a good place to start is the OpenSearch guide for installation

Step 2: Configure OpenSearch

The opensearch.yml file contains configuration details for your OpenSearch server.

The location of your OpenSearch configuration directory varies depending on how you installed OpenSearch. For installations from an archive the configuration is $OPENSEARCH_HOME/config.

To configure your remote OpenSearch server

  1. Locate the opensearch.yml file within the configuration directory of your OpenSearch server.

  2. Add these properties to your opensearch.yml file:

opensearch.yml
action.auto_create_index: ".watches,.triggered_watches,.watcher-history-*"
network.host: 0.0.0.0

Step 3: Secure OpenSearch

Bitbucket communicates to OpenSearch over HTTP using the OpenSearch REST API and can be configured to use basic authentication (a username and password). The OpenSearch provided security plugin can be configured for this purpose.

The minimal distribution of OpenSearch does not contain the OpenSearch security plugin. A custom security solution is required to use the minimal OpenSearch distribution with Bitbucket.

The instructions below describe a simple way to configure the OpenSearch security plugin for the minimum requirements that Bitbucket needs to connect to OpenSearch securely. This is useful if you are migrating from using Elasticsearch with Buckler and want the search server security configuration to work in a similar way, or just want a basic configuration to begin with.

The OpenSearch security plugin also supports many additional features and methods of configuration, which are documented in the OpenSearch security plugin documentation.

Follow the instructions below to secure your remote OpenSearch server with the OpenSearch security plugin. This will involve configuring the OpenSearch security plugin to include a single user in its internal users database called bitbucket with a selected hashed password which has access to the entire search cluster, and a single authentication domain providing basic authentication against the internal users database.

Transport layer TLS

Transport layer TLS is mandatory to use the OpenSearch security plugin (and optionally TLS can also be applied at the REST layer). See the OpenSearch security plugin Configure TLS certificates documentation on how TLS can be configured. Self-signed certificates can be used (provided the plugins.security.ssl.transport.enforce_hostname_verification: false property is set in opensearch.yml) and can be generated using a tool like OpenSSL. Place the generated certificates or keystore/truststore files in the $OPENSEARCH_HOME/config directory.

Security configuration

Replace the configuration files in $OPENSEARCH_HOME/plugins/opensearch-security/securityconfig/ with the following: 

action_groups.yml
_meta:
  type: "actiongroups"
  config_version: 2
audit.yml
_meta:
  type: "audit"
  config_version: 2

config:
  # enable/disable audit logging
  enabled: false
config.yml
_meta:
  type: "config"
  config_version: 2

config:
  dynamic:
    authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: intern
internal_users.yml
_meta:
  type: "internalusers"
  config_version: 2

bitbucket:
  hash: "##REPLACE WITH HASHED PASSWORD##"
  backend_roles:
  - "admin"
  description: "Admin user"
nodes_dn.yml
_meta:
  type: "nodesdn"
  config_version: 2
roles.yml
_meta:
  type: "roles"
  config_version: 2
roles_mapping.yml
_meta:
  type: "rolesmapping"
  config_version: 2

all_access:
  reserved: false
  backend_roles:
  - "admin"
  description: "Maps admin to all_access"
tenants.yml
_meta:
  type: "tenants"
  config_version: 2
whitelist.yml
_meta:
  type: "whitelist"
  config_version: 2

config:
  enabled: false

The important files are config.yml, internal_users.yml, and roles_mapping.yml:

a. config.yml: defines a single authenticator, which is using basic authentication on both HTTP and transport, backed by the internal user database

b. internal_users.yml: creates a single user with username bitbucket (and a provided hashed password, see Generate a password step below), and this user is assigned the admin back-end role

c. roles_mapping.yml: assigns the admin backend role (that the bitbucket user has) the all_access role, which is a pre-defined role by the OpenSearch security plugin that gives access to the entire search server

d. The remaining files are empty placeholders that the OpenSearch security plugin requires.

Generate a password

Generate a hashed password for the bitbucket user using the OpenSearch provided hashing tool $OPENSEARCH_HOME/plugins/opensearch-security/tools/hash.sh. Copy the hashed password into internal_users.yml, replacing the password placeholder.

This password is what Bitbucket will use to connect to OpenSearch.

OpenSearch configuration

To configure OpenSearch security the following properties also need to be added to opensearch.yml:

opensearch.yml
#This property causes the OpenSearch security plugin to configure itself based on
#the security configuration yml files if no security configuration exists in on the
#search server yet (e.g. on first start).
plugins.security.allow_default_init_securityindex: true

#The REST API is locked down by default, even to the 'all_access' roll, so enable
#the 'all_access' role to be able to use it here.
plugins.security.restapi.roles_enabled: ["all_access"]

#Transport layer TLS is mandatory. The certificates need to be manually added to the
#OpenSearch config folder. The example settings below are for X.509 certificates and
#PKCD #8 keys, but a keystore and truststore approach can also be used. Refer to the
#OpenSearch security plugin documentation for more information.
plugins.security.ssl.transport.pemcert_filepath: node.pem
plugins.security.ssl.transport.pemkey_filepath: node-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem

#If the generated certificates are self-signed then hostname verification must be
#disabled.
#plugins.security.ssl.transport.enforce_hostname_verification: false

Start your remote OpenSearch server.

When using plugins.security.allow_default_init_securityindex: true, changing the configuration files in $OPENSEARCH_HOME/plugins/opensearch-security/securityconfig/ after the first startup of OpenSearch will not have any effect, as the files are only used automatically for the default initialisation of the security index.

To modify configuration later, you can either use the securityadmin.sh script or the REST API.

Step 4: Connect OpenSearch to Bitbucket

Once you've configured your OpenSearch server, you then need to connect it to Bitbucket. 

To configure your remote OpenSearch server using the bitbucket.properties file

Once a parameter is set in the bitbucket.properties file, it cannot be edited later from the admin UI. Any changes that need to be made to the OpenSearch configuration, must be made within the bitbucket.properties file and require a restart of Bitbucket to be applied.
  1. Locate the bitbucket.properties file in the <Bitbucket home directory>/shared directory. 
  2. Add the details of your OpenSearch server:

    <Bitbucket home directory>/shared/bitbucket.properties
    plugin.search.config.baseurl=<search server URL> #e.g. http://localhost:9200/
    plugin.search.config.username=<username> #e.g. bitbucket
    plugin.search.config.password=<password> #e.g. the password that was hashed
  3. Start Bitbucket without starting the bundled search server.

    Bitbucket Data Center clustered

    Bitbucket Data Center clustered does not install the bundled search server, so will always start without it.

    Bitbucket Data Center single node (not a service)

    start-bitbucket.sh --no-search

    Bitbucket Data Center single node (as a service)

    Modify your start service script to pass --no-search to start-bitbucket.sh.

    Your remote OpenSearch server is now configured to work with Bitbucket.

Last modified on Oct 23, 2023

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.