Mesh configuration properties
This page describes the configuration properties that can be used to control behavior in Bitbucket Mesh. Create the mesh.properties
file, in the home directory, and add the system properties you need, use the standard format for Java properties files.
Bitbucket Mesh must be restarted for changes to become effective.
Default values for system properties, where applicable, are specified in the tables below.
Authentication
Default value | Description |
---|---|
authentication.allowed-clock-skew | |
2m | Defines the clock skew allowed when validation expiry for signed tokens during authentication. This accounts for clock drift between Bitbucket and Mesh nodes. This value is in SECONDS unless a unit (s, m, h, d) is specified. |
authentication.expiry-interval | |
30s | Defines the amount of time a signed token is valid after it's issued. This only affects outbound requests, such as replication requests from one Mesh node to another. Since tokens are generated right before the RPC they will be used to authenticate, this interval should generally be short. This value is in SECONDS unless a unit (s, m, h, d) is specified. |
Commits
Default value | Description |
---|---|
commit.message.max | |
262144 | Controls the maximum length of the commit message to be loaded when retrieving one or more commits. Commit messages longer than this limit will be truncated. The default limit is high enough to not affect processing for the general case, but protects the system from consuming too much memory in exceptional cases. |
commit.message.bulk.max | |
16384 | Controls the maximum length of the commit message to be loaded when retrieving commits in bulk. Commit messages longer than this limit will be truncated. The default limit is high enough to not affect processing for the common case, but protects the system from consuming too much memory when many commits have long messages. |
Database
Default value | Description |
---|---|
database.pool.idle-size | |
5 | Defines the number of connections the pool tries to keep idle. The system can have more idle connections than the value configured here. As connections are borrowed from the pool, this value is used to control whether the pool will eagerly open new connections to try and keep some number idle, which can help smooth ramp-up for load spikes. |
database.pool.max-size | |
50 | Defines the maximum number of connections the pool can have open at once. |
database.pool.connect-timeout | |
15 | Defines the amount of time the system will wait when attempting to open a new connection before throwing an exception. The system may hang, during startup, for the configured number of seconds if the database is unavailable. As a result, the timeout configured here should not be generous. If no unit is specified (ms, s, m, h) this value is in seconds. |
database.pool.idle-timeout | |
30 | Defines the maximum period of time a connection may be idle before it is closed. In general, generous values should be used here to prevent creating and destroying many short-lived database connections (which defeats the purpose of pooling). If no unit is specified (ms, s, m, h) this value is in minutes. |
database.pool.leak-timeout | |
0 | Defines the maximum period of time a connection may be checked out before it is reported as a potential leak. By default, leak detection is not enabled. Long-running tasks can easily exceed this threshold and trigger a false positive detection. If no unit is specified (ms, s, m, h) this value is in minutes. |
database.pool.lifetime-timeout | |
60 | Defines the maximum lifetime for a connection. Connections which exceed this timeout are closed the first time they become idle and fresh connections are opened. If no unit is specified (ms, s, m, h) this value is in minutes. |
Executor
Controls the thread pool for general asynchronous processing.
Default value | Description |
---|---|
executor.max-threads | |
${scaling.concurrency} | Controls the maximum number of threads allowed in the common |
executor.timeout | |
60 | Controls how long a thread submitting a task to the common This value is in seconds. |
Git
Default value | Description |
---|---|
git.path.executable | |
Defines the path to the git executable. If no value is set, the system searches the runtime user's PATH and standard system paths to find a suitable git executable. Setting a value here is usually not necessary. This is left here purely for documenting how to set an explicit path. |
Git Hooks
Default value | Description |
---|---|
hooks.callback.timeout | |
240 | Controls the maximum time a git hook callback is allowed to execute before being aborted, thereby rejecting the request if the hook can reject the request. When one or more refs are updated and a pre-receive or post-receive request is sent back to the RPC client, this setting controls how long the application will wait for the RPC client to respond. This value is in seconds. |
Git signing
Default value | Description |
---|---|
git.signing.enabled | |
false | Controls whether system created Git objects (such as pull request merge commits) are signed. When this feature is enabled the application will sign and verify system created Git objects using a GPG signing key-pair provided by the control plane. |
Home directory layout overrides
Default value | Description |
---|---|
layout.caches-dir | |
Controls the directory used to store caches, including the pack-objects caches used to cache clones. Files in this directory do not need to be retained across restarts. In cloud deployments, it's perfectly acceptable to use an ephemeral drive for the caches directory. Using a drive that provides good read performance for the caches directory will greatly benefit clone performance. Use an absolute path when configuring this property. | |
layout.tmp-dir | |
Controls the directory used to store temporary files. Files in this directory do not need to be retained across restarts. In cloud deployments, it's perfectly acceptable to use an ephemeral drive for the tmp directory. Use an absolute path when configuring this property. |
Hook Scripts
Default value | Description |
---|---|
hookscripts.gc.interval | |
24 | Defines the number of hours to wait between garbage collection runs for hook scripts. This unit is in hours. The default is 24 hours (1 day). |
hookscripts.gc.prune | |
168 | Defines the minimum age of a hook script before it can be considered for garbage collection. This unit is in hours. The default is 1 week. |
Hosting
Default value | Description |
---|---|
hosting.allow-filter | |
true | Controls whether partial clones, using --filter, are allowed. Partial clones are not cached, and some of the filters offered by Git can be very resource-intensive for the server to apply, so it can sometimes be more efficient to use a normal clone (or a shallow one) instead. Partial clones are enabled by default. |
hosting.atomic-initial-push | |
true | Controls whether initial pushes are treated as an atomic push by the system. This option is enabled by default because it significantly reduces the overhead of coordinating and replicating the initial push. Only disable this option in case of issues with initial pushes. |
Hosting Cache (Pack Cache)
See Scaling for Continuous Integration performance for more information about configuring the Git Cache.
Note: The settings controlled by these properties can be dynamically updated using REST. The REST configuration takes precedence over the configuration in mesh.properties
.
Default value | Description |
---|---|
hosting.cache.expiry.check.interval | |
30 | Controls how frequently expired cache entries are invalidated and removed from the cache. This value is in seconds. |
hosting.cache.eviction.hysteresis | |
1073741824 | When eviction is triggered the amount of disk space requested for eviction is calculated as (eviction.hysteresis + eviction.trigger.free.space - free space under <Mesh home directory>/caches) This value is in bytes. |
hosting.cache.eviction.trigger.free.space | |
6442450944 | Controls the threshold at which eviction is triggered in terms of free space available on disk (specifically under <Mesh home directory>/caches) This value is in bytes. |
hosting.cache.minimum.free.space | |
5368709120 | Controls how much space needs to be available on disk (specifically under <Mesh home directory>/caches) for caching to be enabled. This setting ensures that the cache plugin does not fill up the disk. This value is in bytes. |
hosting.cache.pack.enabled | |
true | Controls whether caching is enabled for git-upload-pack (clone operations). |
hosting.cache.pack.fetch | |
false | Controls whether fetches should be cached in addition to clones. |
hosting.cache.pack.maxCount | |
20 | The maximum number of upload-pack cache entries to retain per repository. If there are more than this configured limit, the least recently accessed entry will be invalidated. |
hosting.cache.pack.partial | |
false | Controls whether partial clones or fetches should be cached. |
hosting.cache.pack.ttl | |
14400 | Controls how long the caches for clone operations are kept around when there no changes to the repository. Caches are automatically invalidated when someone pushes to a repository or when a pull request is merged. This value is in seconds. |
hosting.cache.protocol.http.enabled | |
true | Controls which whether caching over HTTP(S) is enabled. |
hosting.cache.protocol.ssh.enabled | |
true | Controls which whether caching over SSH(S) is enabled. |
Merges
Default value | Description |
---|---|
merge.merge-tree | |
true | Whether to perform merges using |
Metrics
Default value | Description |
---|---|
metrics.tags.application | |
Mesh | Any metrics tags that should be added to all metrics reported from this node can be added as properties under |
management.metrics.export.datadog.enabled | |
false | Datadog metrics configuration. Please set |
management.metrics.export.datadog.api-key | |
YOUR_API_KEY | |
management.metrics.export.datadog.step | |
30s | |
management.metrics.export.dynatrace.enabled | |
false | Dynatrace metrics configuration. Please set |
management.metrics.export.dynatrace.api-token | |
YOUR_TOKEN | |
management.metrics.export.dynatrace.device-id | |
YOUR_DEVICE_ID | |
management.metrics.export.dynatrace.uri | |
YOUR_URI | |
management.metrics.export.influx.enabled | |
false | InfluxDB metrics configuration. Please set |
management.metrics.export.influx.auto-create-db | |
true | Whether to create the Influx database if it does not exist before attempting to publish metrics to it. (Default: true) |
management.metrics.export.influx.batch-size | |
10000 | Number of measurements per request to use for this backend. If more measurements are found, then multiple requests will be made. (Default: 10000) |
management.metrics.export.influx.batch-size.compressed | |
true | Whether to enable GZIP compression of metrics batches published to Influx. (Default: true) |
management.metrics.export.influx.batch-size.connect-timeout | |
1s | Connection timeout for requests to this backend. (Default: 1s) |
management.metrics.export.influx.batch-size.consistency | |
one | Write consistency for each point. (Default: one) |
management.metrics.export.influx.batch-size.db | |
mydb | Tag that will be mapped to "host" when shipping metrics to Influx. (Default: mydb) |
management.metrics.export.influx.batch-size.enabled | |
true | Whether exporting of metrics to this backend is enabled. (Default: true) |
management.metrics.export.influx.batch-size.num-threads | |
2 | Number of threads to use with the metrics publishing scheduler. (Default: 2) |
management.metrics.export.influx.batch-size.password | |
mysecret | Login password of the Influx server. |
management.metrics.export.influx.batch-size.read-timeout | |
10s | Read timeout for requests to this backend. (Default: 10s) |
management.metrics.export.influx.batch-size.retention-policy | |
my_rp | Retention policy to use (Influx writes to the DEFAULT retention policy if one is not specified). |
management.metrics.export.influx.batch-size.step | |
1m | Step size (i.e. reporting frequency) to use. (Default: 1m) |
management.metrics.export.influx.batch-size.uri | |
http://localhost:8086 | URI of the Influx server. (Default: http://localhost:8086) |
management.metrics.export.influx.batch-size.user-name | |
myusername | Login user of the Influx server. |
management.metrics.export.jmx.enabled | |
${jmx.enabled} | Configures whether metrics should be exposed over JMX |
management.metrics.export.jmx.domain | |
metrics | |
management.metrics.export.jmx.step | |
5s | |
management.metrics.export.newrelic.enabled | |
false | NewRelic metrics configuration. Please set |
management.metrics.export.newrelic.account-id | |
YOUR_ACCOUNT_ID | |
management.metrics.export.newrelic.api-key | |
YOUR_API_KEY | |
management.metrics.export.newrelic.step | |
1m | The interval at which metrics are sent to New Relic. See Duration.parse for the expected format. The default is 1 minute. |
management.metrics.export.signalfx.enabled | |
false | SignalFX metrics configuration. Please set |
management.metrics.export.signalfx.access-token | |
YOUR_ACCESS_TOKEN | |
management.metrics.export.statsd.enabled | |
false | StatsD metrics configuration. Please set |
management.metrics.export.statsd.host | |
statsd.example.com | |
management.metrics.export.statsd.port | |
9125 | |
management.metrics.export.statsd.protocol | |
udp | |
throttle.resource.mirror-hosting.duration.tracking.enabled | |
false | Whether to record metrics for how long each mirror-hosting ticket is held. This value will not apply if (available in 8.19.8+) |
throttle.resource.scm-command.duration.tracking.enabled | |
false | Whether to record metrics for how long each scm-command ticket is held. This value will not apply if (available in 8.19.8+) |
throttle.resource.scm-hosting.duration.tracking.enabled | |
false | Whether to record metrics for how long each scm-hosting ticket is held. This value will not apply if (available in 8.19.8+) |
throttle.resource.scm-refs.duration.tracking.enabled | |
false | Whether to record metrics for how long each scm-refs ticket is held. This value will not apply if (available in 8.19.8+) |
Process execution
Controls timeouts for external processes, such as git
.
Default value | Description |
---|---|
process.timeout.execution | |
120 | Configures a hard upper limit on how long the command is allowed to run even if it is producing output. This value is in seconds. Using 0, or a negative value, disables the timeout completely. |
process.timeout.idle | |
60 | The idle timeout configures how long the command is allowed to run without producing any output. This value is in seconds. Using 0, or a negative value, disables the timeout completely. |
Profiling
Default value | Description |
---|---|
profiling.enabled | |
${atlassian.profile.activate:false} | Controls whether profiling should be enabled or not |
profiling.min-frame-time | |
${atlassian.profile.mintime:1} | Defines the threshold time (in milliseconds) below which a profiled event should not be reported. |
profiling.min-trace-time | |
${atlassian.profile.mintotaltime:0} | Defines the threshold time (in milliseconds) below which an entire stack of profiled events should not be reported. |
profiling.max-frame-name-length | |
${atlassian.profile.maxframenamelength:300} | Defines the maximum length of a profiling frame in the profiling log. Setting this value too high can lead to out of memory issues because profiling frames are kept in memory until the request ends. Longer frames will be abbreviated. |
Repair
Exponential backoff between (failed) attempts to repair a repository replica
Default value | Description |
---|---|
repair.backoff.initial-delay | |
30 | Configures the delay between the first repair failure and the next attempt to repair the replica. This value is in seconds. |
repair.backoff.jitter | |
0.05 | Configures the fraction of jitter to apply to the next delay to help spread out delays of concurrent repair operations. The default is 5%, which results in the next calculated delay being increased or decreased by up to 2.5% |
repair.backoff.max-delay | |
1800 | Configures the maximum delay between two repair attempts. This value is in seconds. The default is 30 minutes. |
repair.backoff.multiplier | |
1.2 | Configures the factor by which the delay between two repair attempts should be increased The default value is 1.2, for a ~20% increase between attempts (+/- jitter). |
repair.max-attempts | |
25 | Configures the maximum amount of times repair is attempted if the replica does not become consistent after repair completes, or repair fails because no consistent replicas are available to repair from. |
repair.throttle.limit | |
10 | Configures the maximum number of concurrent repair operations |
repair.throttle.timeout | |
30 | Configures a hard upper limit on how long repair will wait for a repair slot to become available This value is in seconds. Using 0, or a negative value, disables the timeout completely. The default is 30 seconds. |
repair.timeout.execution | |
86400 | Configures a hard upper limit on how long repository repair is allowed to run. This value is in seconds. Using 0, or a negative value, disables the timeout completely. The default is 24 hours. |
repair.update-ref.batch-size | |
20000 | Configures the maximum amount of refs to update in a single git update-ref operation as part of repair. If repair needs to update more than the configured limit, multiple git update-ref operations will be used. |
Replication
Default value | Description |
---|---|
replication.quorum-mode | |
majority | Controls how replica votes are applied when replicating transactions. Available modes are: - The default is to require a majority vote. |
Storage Management
Default value | Description |
---|---|
storage.partition-cleanup-delay | |
3600 | Determines the delay between when a topology update occurs and the storage cleanup job running. The cleanup job will delete partitions on disk that are not hosted by this Mesh node anymore. The default delay is 1 hour specified in seconds. This value is in seconds. |
storage.temp-cleanup-interval | |
24 | Determines the interval used for how frequent the clean up of the tmp directory on the Mesh node job is run. The default interval is 1 day specified in hours. |
storage.temp-expiry | |
72 | Determines the interval used to calculate whether a file or directory is stale for it to be deleted on the Mesh node in the tmp directory. The default interval is 3 days specified in hours. |
Throttling
These properties define concurrent task limits for the ThrottleService, limiting the number of concurrent operations of a given type that may be run at once. This is intended to help prevent overwhelming the server hardware with running processes. Two settings are used to control the number of processes that are allowed to process in parallel: one for the web UI and one for 'hosting' operations (pushing and pulling commits, cloning repositories).
When the limit is reached for the given resource, the request will wait until a currently running request has completed. If no request completes within a configurable timeout, the request will be rejected.
The underlying machine-level limits these are intended to prevent hitting are very OS- and hardware-dependent, so you may need to tune them for your instance. When hyperthreading is enabled for the server's CPUs, for example, it is likely that the default settings will allow sufficient concurrent operations to saturate the I/O on the machine. In such cases, we recommend starting off with a less aggressive default on multi-cored machines; the value can be increased later if hosting operations begin to back up.
Additional resource types may be configured by defining a key with the format throttle.resource.<resource-name>
. When adding new types, it is strongly recommended to configure their ticket counts explicitly using this approach.
Default value | Description |
---|---|
scaling.concurrency | |
cpu | Allows adjusting the scaling factor used. Various other CPU/throttling dependent properties are defined in terms of this setting, so adjusting this value implicitly adjusts those. Some examples:
The default value, |
throttle.resource.git-lfs | |
80 | Limits the number of Git LFS file transfer operations which may be running concurrently. This is primarily intended to prevent Git LFS requests from consuming all available connections thereby degrading UI and Git hosting operation. This should be a fraction of the maximum number of concurrent connections permitted by Tomcat. |
throttle.resource.git-lfs.timeout | |
0 | Controls how long threads will wait for Git LFS uploads/downloads to complete when the system is already running the maximum number of concurrent transfers. It is recommended this be set to zero (i.e. don't block) or a few seconds at most. Since waiters still hold a connection, a non-zero wait time defeats the purpose of this throttle. This value is in seconds. |
throttle.resource.mirror-hosting | |
2*${scaling.concurrency} | Limits the number of SCM hosting operations served to mirrors, which may be running concurrently. This limit is intended to protect the system's CPU and memory from being consumed excessively by mirror operations. Note that dynamic throttling is not supported for mirror hosting operations. |
throttle.resource.mirror-hosting.timeout | |
3600 | Controls how long threads will wait for mirrors hosting operations to complete when the system is already running the maximum number of git commands. This value is in seconds. |
throttle.resource.scm-command | |
50 | Limits the number of git commands, such as: |
throttle.resource.scm-command.timeout | |
2 | Controls how long threads will wait for SCM commands to complete when the system is already running the maximum number of SCM commands. This value is in seconds. |
throttle.resource.scm-hosting.timeout | |
300 | Controls how long threads will wait for SCM hosting operations to complete when the system is already running the maximum number of SCM hosting operations. This value is in seconds. |
throttle.resource.scm-hosting.strategy | |
adaptive | Specifies the strategy for throttling SCM hosting operations. Possible values are 'adaptive' and 'fixed'. If 'fixed' is specified, throttle.resource.scm-hosting.fixed.limit is used as the fixed upper limit on the number of concurrent hosting operations. If 'adaptive' is specified, the maximum number of hosting operations will vary between throttle.resource.scm-hosting.adaptive.min and throttle.resource.scm-hosting.adaptive.max based on how many hosting operations the system believes the machine can support in its current state and given past performance. If any configured adaptive throttling setting is invalid and reverts to a default but this conflicts with other correctly configured or default settings, the throttling strategy will revert to 'fixed'. E.g. this will occur if |
throttle.resource.scm-hosting.adaptive.limit.min | |
1*${scaling.concurrency} | When the adaptive strategy is enabled for throttling SCM hosting operations, this sets the lower limit on the number of SCM hosting operations, meaning pushes and pulls over HTTP or SSH, which may be running concurrently. Setting a lower limit provides a way to specify a minimum service level for SCM hosting operations regardless of what the adaptive throttling heuristic believes the machine can handle. There is limited support for mathematical expressions; +,-,*,/ and () are supported. |
throttle.resource.scm-hosting.adaptive.limit.max | |
4*${scaling.concurrency} | When the adaptive strategy is enabled for throttling SCM hosting operations, this sets the upper limit on the number of SCM hosting operations, meaning pushes and pulls over HTTP or SSH, which may be running concurrently. This is intended primarily to prevent pulls, which can be very memory-intensive, from exhausting a server's resources. Note that if the machine does not have sufficient memory to support this default value or an explicitly configured value, a smaller value will be chosen on startup. Adaptive throttling will vary the total tickets There is limited support for mathematical expressions; +,-,*,/ and () are supported. |
throttle.resource.scm-hosting.adaptive.cpu.target | |
0.75 | When the adaptive strategy is enabled for throttling SCM hosting operations, this sets the target CPU utilisation for the machine (across all processors) which the system takes into consideration when calculating the current throttling limit. This value represents a trade-off: higher numbers may boost raw throughput of hosting operations, but at the expense of overall system responsiveness for all users. Increasing the target too high or too low brings diminishing returns. This must be a value between 0.0 and 1.0 and is a percentage of the total available CPU power across all cores |
throttle.resource.scm-hosting.fixed.limit | |
1.5*${scaling.concurrency} | When the fixed strategy is enabled for throttling SCM hosting operations, this limits the number of SCM hosting operations, meaning pushes and pulls over HTTP or SSH, which may be running concurrently. This is intended primarily to prevent pulls, which can be very memory-intensive, from exhausting a server's resources. There is limited support for mathematical expressions; +,-,*,/ and () are supported. |
throttle.resource.scm-refs | |
8*${scaling.concurrency} | Limits the number of ref. advertisement operations, which may be running concurrently. Those are throttled separately from clone operations as they are much more lightweight and much shorter so we can and should allow many more of them running concurrently. |
throttle.resource.scm-refs.timeout | |
120 | Controls how long threads will wait for ref. advertisement operations to complete when the system is already running the maximum number of ref. advertisement operations. This value is in seconds. |
gRPC
Default value | Description |
---|---|
grpc.server.address | |
0.0.0.0 | The address the gRPC services should bind to. The default value binds to all hosts. |
grpc.server.port | |
7777 | The port to use for the gRPC services |
grpc.server.ssl.cert-chain-path | |
The location of the file containing the full certificate chain, when SSL/TLS is used. Default is empty, which means SSL is disabled. Setting a value for this property will only have an effect if | |
grpc.server.ssl.private-key-path | |
The location of the file containing the private key, when SSL/TLS is used. Default is empty, which means SSL is disabled. Setting a value for this property will only have an effect if |