Federating Jira - managing multiple instances

If your organization uses multiple instances of Jira to handle the load of your users, we recommend that you consolidate those instances into a single Jira Data Center instance. This will provide you with the administrative features, interoperability, high availability, and performance every large organization needs from Jira. 

However, we recognize that consolidation and migration might not be a viable option for some organizations. If you're one of them, then you may need to keep your instances federated – this means keeping them autonomous, yet connected and interoperable. This guide explains the range of features and practices to help keep your federated instances healthy.

What leads to a federated environment

First of all, let's look at the circumstances that lead to a federated environment. When talking to our customers, we can identify four main reasons you may wish to run multiple instances of Jira:

• Bottom-up growth of Jira: Due to Jira's low price point and practical value, Jira often starts in a single team and then spreads throughout an organization with new teams spinning up their own server instances.

• Mergers & acquisitions: Through acquiring or merging with other organizations, a company can find itself managing several Jira servers.
• Autonomous IT organizations: Different departments within an organization may run their own IT organizations. This leads to parallel Jira systems with possible integration at a later stage once cross-divisional processes and collaboration is encouraged.
• Intentional federation set up: One, or many, reasons listed below could lead to an intentional set up of federated Jira systems.

Reasons to set up a federated environment

Delving into the intentional set up of dedicated instances, we can identify several reasons for doing so by summarizing the practices of our customers, partners and our own IT organization:

• Public / Private Jira: Jira offers the functionality to make specific content only accessible to identified groups through permission schemes and user groups. However, many customers, including Atlassian itself, prefer to run internal content (e.g. development) on a separate instance inside the firewall. A dedicated instance is then set up to host external content (e.g. support / public feature requests) and make it accessible to customers and partners.
• Different grades of complexity: Let's look at an example where a Jira instance has a single project that targets a minority of users, yet is responsible for the majority of complexity, such as custom fields, workflows and customizations. To prevent this complexity from affecting the overall performance and stability of the entire Jira instance, it makes sense to migrate that project to a dedicated Jira server.
• Access to new features vs stability: An organization might have a situation where a specific teams' productivity is heavily influenced by their access to new features (e.g. new Jira Software features for Agile teams), while other parts of the organization have implemented stable, specific workflows and value system stability over new features. In this case, it might be a viable option to run a stable, conservative, production instance while operating a separate productive Jira server for early adopters.

• External Customer requirement: Some customers might require an organization to run their projects and host their data on systems that are physically separated from the rest of their infrastructure. Such requirements can be the case with classified projects, such as defense contracting.
• Different Configuration or Settings: Jira is famous for its configuration capabilities offering schemes for almost every concern of the application. This enables one instance to host a myriad of different projects. This is useful in cases where a project requires a dedicated instance so that it can have its own branding, or different settings. These might include separate priorities or resolutions, a different set of administrators, use of time tracking or not, and everything else that is a global setting in Jira. 
• Legal requirements: Legal concerns, such as data privacy laws, jurisdiction over servers or government surveillance might lead to the decision to have dedicated servers in separate jurisdictions for specific Jira projects. An example of such a case would be the inability of a European company to store private data of European Union citizens on servers that are hosted in countries outside the EU that do not offer data privacy laws with the same protection of individual rights. 
• Jira as an internal service: Some of our large customers offer Jira internally as a service. They have a standard configuration that they then clone for each business unit that requests a Jira system. Each system then has its own BU-specific administrators while the central IT organization maintains system administration competency. 
• Spreading Risk / Downtime: Larger instances take longer to upgrade. Therefore, one possibility to reduce unavailability of Jira services to all users is to split and thereby reduce the size of instances. That way, an upgrade downtime of eight hours could then be split and turned into two-hour downtimes of four Jira systems with each downtime affecting less users. 

Recommendations for managing multiple Jira instances

Federation of Jira instances is quite common among enterprise customers. Past research by Atlassian counted hundreds of customers that run three or more Jira servers concurrently within their organization. As a result, customers with federated environments are not alone, but part of a larger community. Atlassian has therefore developed and is continuing to implement features that make management and interoperability of federated Jira instances easier for customers.

Federation features

The ability to have Atlassian products work together is one of the strongest features of Jira, Confluence, and the entire Atlassian stack. One of the best examples of this is the possibility to dynamically list Jira issues within a Confluence page. This same functionality also enables the integration of autonomous Jira instances.

The Jira instances, thereby, do not need to run the exact same version as long as all versions contain the Application Links functionality. This makes interoperability in a heterogeneous landscape (e.g. see reasons to federate, "Access to new features") a lot easier.

In a nutshell, the goal of application links is to make inter-instance linking and integration of issues as easy as inter-project integration within the same instance! So after an application link between your Jira instances is set up, the integration functionality described below becomes available.

Unified activity streams

Activity streams not only show updates from the instance you are on but also from connected Jira or other Atlassian product instances (e.g. Confluence or Fisheye). For example, two companies working together on a project and linking up their Jira instances could be updated with new status changes of work at their partner's Jira through a unified activity stream.

Unified reporting dashboards

Gadgets that draw their data from another linked and trusted Jira instance can be added to a Jira dashboard. This enables a unified reporting overview of all relevant instances. Next to the standard gadgets, you can also add additional gadgets. The applications are endless – from just one gadget that reports on a specific metrics of another project in another instance, to having one central instance in your federation that runs reporting of all projects in all Jira systems!

Remote issue links

Linking issues is crucial to connecting relevant Jira issues. With remote links, you can not only track dependencies and other relations of issues to other local projects, but to remote projects in other instances as well.

Application navigator

You do not have to remember the addresses of your Atlassian products any more. The Application Navigator header allows for fast switching between Jira instances, as well as other Atlassian products working together. 

Jira Issue Copy (by Atlassian Labs, not supported)

Atlassian Labs has developed the Jira to Jira Issue Copy, which allows users to copy an issue from one Jira instance to another. It is currently still in Atlassian Labs and available on the Atlassian Marketplace. It has a cleaner and easier to understand user interface, and supports more custom configurations and enhancements in the user matching algorithm. Note that this add-on is not supported by Atlassian, and although it may work for you now, we don't recommend building it into a process that you rely on due to it's Atlassian Labs status.

Federated search

Although searching across multiple instances is not part of an out-of-the-box Jira installation, you can achieve it through ALM's Jira Client. This app is a desktop application that was designed to enable offline work with Jira. It lets you establish connections to an unlimited amount of Jira instances and once this connection has been established, search queries are executed automatically across all the various systems.

Administering federated instances

The following sections describe Jira features that you can use to efficiently administer a federated Jira instance.

Central user management

Jira offers multiple ways to have a single-sourced user management, group definition and authentication (SSO) that multiple Jira instances or even other Atlassian products can draw from:

• Crowd: Atlassian's dedicated product to provide centralized identity management with single sign-on for an organization's applications and developer tools. See the chapter on Connecting to Crowd for User Management for more instructions. In case you want to migrate user directories from Jira to Crowd, see Importing Users from Jira to Crowd.
• LDAP: Is the de facto standard on storing and managing distributed directory information. See the Jira documentation on how to  connect to an LDAP Directory.

For an overview of different scenarios on how multiple Jira systems and/or other Atlassian products could interoperate and accomplish centralized user management, see illustrated Diagrams of Possible Configurations for User Management.

Permission scheme harmonization

The ability to have a single source for user groups also facilitates the harmonization of permission schemes across multiple systems. Groups such as Project Administrators, System Administrators, Internal, External, etc. can be centralized and then assigned to Jira's local permission schemes.

Workflow sharing

The Workflow Sharing feature allows you to share your team's workflow with other teams in your organization on different Jira instances, or external parties in other organizations via the Atlassian Marketplace. This feature allows you to easily use workflows that other people have published, or to move a workflow from staging to production in your own organization. It is also provided as an app for older Jira releases.

Migrate, merge and split

In a federated environment, multiple projects run on multiple instances of Jira. In certain circumstances, it might be desirable to change the federation landscape by altering the number of instances involved or moving projects between them. This can be done by merging and splitting instances as well as through project migration.  

• Migrate a project to another instance - CSV import: If you only want to migrate one project, a quick way is to use the CSV Importer:

1. 1. Create the project in the target instance. 
   2. Export the issues in the source project to a CSV Format. 

   3. Import it into the target system using the CSV importer and use the wizard to match meta-data to the target instance configuration.

      Note that a CSV import will only migrate the issue data itself. App data stored in active objects will not be carried over.

      For more information, please see Importing Data from CSV.

• Merge / Migrate - project import: One way to migrate a single project is to use the export function in the source system and then use project import in the target Jira instance. See Restoring a Project from Backup. 
• Splitting an instance - full XML backup restoration: Using the restoration feature, it is possible to clone a Jira instance through a backup/restore procedure. For more detailed instructions, see Splitting Jira applications. 

1. 1. Set up a new Jira Data Center with the same version of the existing system. 

      This will require a Data Center license. 

   2. Perform a full backup of the source system. 

   3. Import the backup into the new Jira instance.
   4. Choose which duplicate projects to remove from the source and target instances.

   5. Please note that this option only applies if the target instance is empty.

      Although it is possible to run multiple Jira instances on the same server, dedicated hardware / VM is recommendable for enterprise use of Jira.

• Splitting an instance - DB / Filesystem level: Some apps may use the Jira database in a way that can not be exported through the standard XML export.
  
  • Set up another Jira instance and replicate the application server, database and installation / home directories. For detailed instructions, see staging environment. 
  • The steps here are essentially the same as the above: Splitting an instance - full XML backup restoration, however this time use your native database backup tools.

Configuration management

The more Jira instances a federated system has the more there is a need for central administration of these systems, we see clients achieving configuration management by implementing various methods:

• Version Control System: A VCS like Subversion or GIT can be used to keep track of changes to files such as configuration XMLs or templates and can then be deployed to a staging environment or production system respectively. This approach provides the ability to roll-back changes and having one central overview of all configuration files in all instances.
• Configuration Management Tool: Many of our customers use configuration management software to keep track of their IT systems, versions, etc. An example would be Puppet in combination with a puppet-Jira app. 
• Continuous Deployment: CI applications like Bamboo can be used to automatically or manually deploy configuration changes to a Jira's staging environment/ production system.
• Jira Workflow Sharing Plugin, by Atlassian Labs (not supported). Also mentioned above in the 'Merge/Split' section. Can be used to centrally maintain and configure workflows and share them across multiple Jira systems.

Agile boards in distributed  environment

If you need to work in distributed environment WatchTower for Jira can help you to consolidate issues from remote Cloud and Server Jira instances into one single agile board. WatchTower gives you a single point of control to work with issues from remote instances on agile board in your Jira. Watchtower saves you time on context switching and gathering complete picture for several projects spread over federated environment. 

Synchronization of projects across instances

In some scenarios,when Jira systems are integrated and deal with similar entities, it might be desirable to have actions in one system affect another. This is not an out-of-the-box feature but can be achieved through the use of apps and scripting:

• Backbone Issue Sync for Jira, supported by K15t 
  The Backbone Issue Sync for Jira  synchronizes issue data between different Jira instances. Synchronize Jira project data across departmental and B2B borders with ease, flexibility, and confidence. 

• Power Scripts - Jira Workflow Automation, supported by Appfire

  The Power Scripts app provides a new scripting layer for Jira that enables administrators to create new functionality. It is discussed in the marketplace as being a solution for synchronization and cross-system workflows for federated Jira environments. 
• Enterprise Mail Handler for Jira (JEMH), supported by The Plugin People
  The Enterprise Mail Handler for Jira is an app which was originally designed to provide advanced functionality to handle incoming emails. This includes the interaction with users that do not have a Jira account or users that use several email aliases. Through its project and field mapping, it can also be used to synchronize and update issues between different instances. For example, you can automatically create an issue in instance A if the issue transitions to a particular state in instance B. Setting up such automation however should be thoroughly tested as there can be occurrences of infinite loop if issues are updated at the same time on both instances.
• ScriptRunner,  supported by Adaptavist
  ScriptRunner is used by many customers to add further scripting functionality to Jira. The app supports input in languages including Groovy, Python (jython), Ruby (JRuby) and JavaScript. 

Upgrading multiple instances

In order to avoid manual repetition of the upgrade instructions given in the Jira administration manual, there are ways to partially or fully automate the process:

• Automated Backup: You can export data from Jira using the Jira Command line interface. (The Jira CLI can also be quite useful for automated tests).
• Symlinks: Instead of working only with the installation directory, a system administrator could maintain a directory for each Jira version. Once the new version has been installed and modified, a symbolic link could be modified to point to the folder of the new version. This is a practice that has proven itself at Atlassian internally.

For Jira instances with high availability requirements, it is also recommended to use a staging server setup for Jira. You do not require a separate license for a staging server. 

Planning HA / failover for multiple instances

Federating Jira instances on one Jira Data Center will enable you to set different availability requirements for your instances. You can certainly leverage the same tools for many instances such as the load balancer (given it runs in HA mode as well) for multiple Jira instances.

Refer to the Failover for Jira Guide for more information.

More resources