Set up a small-scale Hipchat Data Center cluster
Hardware requirements for a small-scale Hipchat Data Center deployment
- The reverse proxy host can be small: 2 CPU cores 1GHz (or faster), with 2GB of RAM.
The Hipchat application node must have 8 CPU cores 2.8GHz (or faster),16GB RAM, and a minimum of 40GB of (non-RAM) storage space.
The data stores node must have 8 CPU cores 2.8GHz (or faster), 16GB RAM, and at least 60GB of non-RAM storage.
If you are migrating from a Hipchat Server deployment, you might need to increase the data store node's available storage based on the number and size of messages in your exported server data.
Network configuration requirements
- Hipchat nodes must be deployed on a private network. You should only be able to access the nodes through a reverse proxy or jumpbox.
- Your private network should only allow inbound connections on port 80 (HTTP), 443 (HTTPS), and 22 (SSH). (See below for details.)
- SSL must be terminated at a load balancer or reverse proxy.
- Hipchat nodes must have unrestricted network access to any other Hipchat nodes in the cluster.
- Hipchat nodes must have unrestricted network access to the data stores.
- Hipchat nodes also require access to ports 53 for DNS and 123 for NTP, and may require additional access to enable optional features such as email notifications, Add-ons, Hipchat Video, and mobile notifications. You might need to write additional DNS-based firewall or proxy rules to allow this access. See the Open and Outgoing ports for more information.
The private network on which you deploy should allow outbound access or have a forward proxy. If your organization uses a DMZ, you can deploy the reverse proxy there.
Open and outgoing ports
Depending on which Hipchat features you choose to enable, you may need to unblock or write firewall rules for the following ports. If you are using a forward proxy, you can use it to access most of these services without writing rules.
Feature | Optional or required? | Port/Protocol | Notes |
---|---|---|---|
DNS | Required | 53 TCP/UDP | Used for DNS resolution of hostnames and to set up SSL trust. |
NTP (Network Time Protocol) | Required | 123 TCP/UDP | Keeps the cluster members on the same schedule. |
Email notifications | Optional (recommended) | 25 TCP | If your SMTP server is accessible by the Hipchat nodes from inside the network, you shouldn't need to open additional ports. |
Native mobile notifications | Optional | 443 TCP to barb.hipch.at | You can whitelist all of |
Hipchat video | Optional | Server: 443 TCP to Clients: 10000 UDP and 443 TCP to | The Hipchat nodes require one-time access to the central Video server to register themselves and create a trust keypair. Clients require additional open access on ports 10000 (with 443 as a fallback) to access the video service. |
Add-ons | Optional | 443 TCP to marketplace.atlassian.com and marketplace-cdn.atlassian.com | Used for retrieving Add-Ons listings from the Atlassian Marketplace. Add-ons may require additional access to function correctly. |
Analytics reporting ("Phone home") | Optional (recommended, please!) | 443 TCP to
https://hipchat-server-stable.s3.amazonaws.com/ohaibtf.html
| We use the statistics reported to these servers to help make Hipchat better! |
Cluster configuration requirements
Hipchat node configuration
- The node must have a static IPv4 address. (This IP should not be accessible by the public internet.) See the official Ubuntu network configuration documentation for more information.
- The node must be configured in the UTC timezone, and must keep the time synchronized using NTP. (This requires access to port 123 over TCP/UDP.)
The node must have a unique hostname. This name must be unique among all members of the cluster. (This requires access to port 53 over TCP/UDP.)
Hostname hints
For clarity in logs and troubleshooting, the host name should also be different from the public DNS entry used to access the cluster. (We know it's tempting to just call it
hipchat
, but resist!)You can set the hostname on a Hipchat node by logging in with SSH and using the following command:
sudo dont-blame-hipchat -c "hostnamectl set-hostname hipchat1.example.com"
If your DNS server does not resolve the hostname you set, you can also add an entry similar to the one below to the
/etc/hosts/
file on each node.127.0.0.1 hipchat1.example.com
Reverse proxy configuration
You may use any reverse proxy that meets the following requirements:
Must be configured for HTTP persistent connections (also called
http-keepalive
).- Must support HTTPs endpoints and SSL offloading.
This will terminate SSL for the cluster, so that all communication within the cluster is unencrypted.
- Must be configured to forward traffic from port 443 to port 80.
- Must be configured with a DNS record to establish SSL trust, and so the chat clients can access it.
- Must be configured with an SSL certificate for connections on port 443.
If you will be using publicly hosted Apps, add-ons, or integrations, the service should be accessible from the public internet so that these external services can send responses back to the cluster.
See the Hipchat Knowledge Base article for more detailed instructions on setting up a basic NGINX reverse proxy for a small-scale Hipchat Data Center.
Redis cache requirements
- Deploy a Redis cache using version 3.2.
The official Redis basic deployment documentation is here, however you may also use a package manager for your environment. - Use the default Redis port of
6379
. - Record the address of the instance.
- For security purposes, we recommend that you enable authentication, and then change the default password. (Make sure you record these credentials for later.)
Enable persistence by setting the following configuration values:
appendonly no appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-min-size 64mb auto-aof-rewrite-percentage 100 stop-writes-on-bgsave-error yes rdbcompression yes maxclients 10000 repl-timeout 60
See the Hipchat Knowledge Base article for more detailed instructions on setting up a basic Redis cache for Hipchat Data Center.
NFS volume requirements
- Must be NFS v4.
- Must be a minimum of 40GB.
- This volume must be accessible anonymously (with read and write permissions) by the Hipchat nodes.
- Root squashing must be disabled. (Learn more.)
See the Hipchat Knowledge Base article for more detailed instructions on setting up a basic NFS server for Hipchat Data Center.
Postgres database requirements
- Deploy a Postgres instance using version 9.5. Use the the official Postgres installation guides found here.
- The instance should be configured in the UTC timezone, and must use NTP to stay synchronized with the rest of the cluster.
Record the IP or DNS address of the host, or an endpoint that can be used to access it (such as a dedicated load balancer for the database).
- Set the database to use UTF-8.
Set the
max_connections
to 1000.
See the Hipchat Knowledge Base article for more detailed instructions on setting up a basic Postgres database for Hipchat Data Center.
Once the Postgres instance is running, create a database and service user:
- Create a database on the Postgres instance. The database name must:
- start with a letter
- be between 8 and 30 characters
- only include letters, numbers, dashes, periods, and underscores
- Create a user to access the database. (Do not use the Postgres SUPERUSER account.)
- Make sure that the new user has
GRANTS ALL
access to the database you just created.
For clarity, we recommend putting the word 'hipchat' in the database name, and in the name of the user that you create to access it. This will help you if you ever need to troubleshoot.
Set up the Hipchat VM
In this step you'll create the virtual machine using the OVA file, and configure it.
Download and deploy the Hipchat OVA
- Create the VM on your VMware host. This process may vary depending on your VMware environment and your organization's standard procedures.
Load the OVA file to create the new VM.
If you're deploying to a VMware instance that has access to the internet, you can sometimes download the binary directly using the URL in the release notes.
Otherwise, you can download the file locally, and then transfer it to the VMware host to deploy the VM. Links to the binaries for both the Production and Beta versions are in the Hipchat Data Center release notes.If you're having trouble downloading the Hipchat OVA file using your browser, you can use a command like
curl
orwget
.$ curl https://s3.amazonaws.com/hipchat-server-stable/dc/HipChat.ova --output HipChat.ova $ wget https://s3.amazonaws.com/hipchat-server-stable/dc/HipChat.ova
Configure the Hipchat VM network
Once you've deployed the VM, you'll configure networking.
From the VM's console:
Edit the
etc/network/interfaces
file. (You can learn more about this file in the official Ubuntu 14.04 network configuration documentation.)
You can use the following command to edit it in vim, but you can use another text editor if you prefer.vim etc/network/interfaces
Change the
iface eth0 inet
line fromdchp
tostatic.
If you're using vim, you'll need to press i to get into "insert" mode which allows you to type.
Add a line for
address
, and set its value to the IP address you allocated for your Hipchat node. (If there was already anaddress
line, replace the existing/default IP with the one you'll be using.)If they're not already included, make sure you have lines for
netmask
andgateway
. (These are usually the same for any server on your network.)
When you're done, your file should look something like this:auto eth0 # turn on automatically iface eth0 inet static # set this to static address 10.0.0.100 # the IP address you allocated for this Hipchat node netmask 255.255.255.0 # network mask gateway 10.0.0.1 # outgoing traffic
Save the file.
If you're using vim, you'll need to press Esc to exit insert mode, then type colon ( : ) and x.
Next run the following two commands to restart networking and apply your changes.
sudo ifdown eth0 sudo ifup eth0
Set the node hostname
The node must have a unique hostname, which must be unique among all members of the cluster. (We know it's tempting to just call it hipchat
, but resist!)
The Hipchat node's name should also be different from the public DNS entry used to access the cluster. (The node requires access to port 53 over TCP/UDP to resolve DNS.)
sudo dont-blame-hipchat -c "hostnamectl set-hostname hipchat-worker-1.example.com"
If your DNS server does not resolve the hostname you set, you can also add an entry similar to the one below to the /etc/hosts/
file on each node.
127.0.0.1 hipchat-worker-1.example.com
Optional - Set custom NTP servers
The Hipchat node must be configured in the UTC timezone, and must keep the time synchronized using NTP. (This requires access to port 123 over TCP/UDP.)
If your deployment has access to the internet, you can skip this configuration step and use the default Atlassian NTP servers provided. (These are 0.atlassian.pool.ntp.org, and 1.atlassian.pool.ntp.org
)
If your environment does not have access to the internet, or if your organization runs its own NTP servers and you wish to use them, you should configure your Hipchat node to use an NTP service inside your network.
To change the NTP servers use the following command. Replace "time1.example.com,time2.example.com
" with a comma-separated list of your own NTP servers.
hipchat service --ntp-servers time1.example.com,time2.example.com
What's next? - The Hipchat Data Center deployment process
Once you've deployed the component services for your Hipchat Data Center cluster, you'll follow the instructions at Configure Hipchat Data Center nodes, then Configure optional Hipchat Data Center features, and Verify and troubleshoot your Hipchat Data Center deployment to complete the process. Then allow users to get online and get chatting!