Migrate to warm standby in Bamboo

If you're already using Bamboo Data Center in a cold standby cluster configuration, use these steps to migrate to the new warm standby configuration after upgrading to Bamboo 9.5.0 or higher. The local home directory on every node in the cluster now contains a cluster-node.properties file that you need to edit before running your Data Center cluster for the first time.

To migrate to warm standby:

1. Go to the local home directory of a node in your cluster and open the cluster-node.properties file in a text editor.
2. Add a new node.hostname property and set it to the node-specific hostname or IP address reachable from the other nodes in the cluster.
3. Optionally, change the default internal node communications port (9090) by adding the node.internal.communication.port property and setting it to a port number that is reachable from the other nodes but not available to the outside world.
4. If you want the underlying gRPC communication to use a proxy, set the bamboo.enable.grpc.via.proxy property to true and remove any references to your Bamboo nodes from the http.nonProxyHosts property. By default, gRPC communication isn't proxied.

5. Optionally, configure any additional system properties.

   Additional system properties

   System property	Default value	Description
   bamboo.physical.queues.per.node.queue	8	The number of physical queues under a single virtual per node queue. The virtual per-node queue is the place where items waiting for propagation to other nodes are stored in order to synchronize their internal states. IMPORTANT: This value can be increased at any time. However, if this value is decreased, then it is necessary to shut down all Bamboo nodes, apply the new value, and start the nodes again.
   bamboo.per.node.physical.queue.max.size	-1	The maximum number of elements in a single physical queue under virtual per node queue. If -1, there is no limit.
   bamboo.per.node.physical.queue.max.used.bytes	-1	The maximum size in bytes of a single physical queue under virtual per node queue. If -1, there is no limit.
   bamboo.grpc.server.threads.number	6	The number of threads to use for the gRPC server scheduler.
   bamboo.cross.nodes.events.grpc.client.threads.number	6	The number of threads to use for the gRPC client scheduler shared across all channels. This client scheduler is used for handling the cross-node event requests.
   bamboo.peer.to.peer.grpc.client.threads.number	2	The number of threads to use for the gRPC client scheduler shared across all channels. This client scheduler is used for handling peer-to-peer requests.
   bamboo.per.node.dispatchers.threads.number	4	The number of threads the per node queue dispatchers' executor will use. The executor is responsible for grabbing the items from the queues and dispatching them to the remote nodes. The gRPC client calls itself are executed on a separate executor.
   bamboo.node.alive.watchdog.interval.seconds	20	The interval (in seconds) at which the node alive watchdog checks if the node is still alive and operational.
   bamboo.cluster.heartbeat.alive.timeout.seconds	300	The duration (in seconds) after which a node is considered dead if no heartbeat is received.
   bamboo.cluster.heartbeat.job.interval.seconds	30	The interval (in seconds) between two heartbeat jobs execution. The job contains the logic responsible for writing the heartbeat of the current node and refreshing the internal cache of the live nodes state.
   bamboo.optimistic.locking.enabled	auto	Optimistic locking mode. When false mechanism is disabled, when true enabled, when auto enabled only when at least two nodes exist.
   bamboo.primary.node.lock.timeout.seconds	120	How long (in seconds) does the Bamboo secondary node wait before deciding that primary node is no longer available.
   bamboo.plugin.system.lock.timeout.seconds	120	How long (in seconds) does the Bamboo plugin system wait before deciding that a taken lock expired and will be taken anyway. It helps to prevent starvation in case of node failure while a lock is taken.
   bamboo.cluster.info.cache.ttl	120	A time to live in seconds for the cluster info cache. The cluster info contains such information as the current state of the cluster, such as whether it is running, paused, etc.
   bamboo.enable.grpc.via.proxy	false	Enables gRPC communication through a proxy. By default, gRPC communication bypasses the proxy.

6. Repeat steps 1 through 4 for the remaining nodes in the cluster.
7. Start all the nodes in the cluster. The node you start up first becomes the primary node, and all the others will become the secondary nodes.