Federating JIRA - Managing Multiple Instances
Reasons to set up a federated environment
Delving into the intentional set up of dedicated instances, we can identify several reasons for doing so by summarizing the practices of our customers, partners and our own IT organization:
- Public / Private JIRA: JIRA offers the functionality to make specific content only accessible to identified groups through permission schemes and user groups. However, many customers, including Atlassian itself, prefer to run internal content (e.g. development) on a separate instance inside the firewall. A dedicated instance is then set up to host external content (e.g. support / public feature requests) and make it accessible to customers and partners.
- Different grades of complexity: Let's look at an example where a JIRA instance has a single project that targets a minority of users, yet is responsible for the majority of complexity, such as custom fields, workflows and customizations. To prevent this complexity from affecting the overall performance and stability of the entire JIRA instance, it makes sense to migrate that project to a dedicated JIRA server.
- JIRA Software features for Agile teams), while other parts of the organization have implemented stable, specific workflows and value system stability over new features. In this case, it might be a viable option to run a stable, conservative, production instance while operating a separate productive JIRA server for early adopters. Access to new features vs stability: An organization might have a situation where a specific teams' productivity is heavily influenced by their access to new features (e.g. new
- External Customer requirement: Some customers might require an organization to run their projects and host their data on systems that are physically separated from the rest of their infrastructure. Such requirements can be the case with classified projects, such as defense contracting.
- Different Configuration or Settings: JIRA is famous for its configuration capabilities offering schemes for almost every concern of the application. This enables one instance to host a myriad of different projects. This is useful in cases where a project requires a dedicated instance so that it can have its own branding, or different settings. These might include separate priorities or resolutions, a different set of administrators, use of time tracking or not, and everything else that is a global setting in JIRA.
- Legal requirements: Legal concerns, such as data privacy laws, jurisdiction over servers or government surveillance might lead to the decision to have dedicated servers in separate jurisdictions for specific JIRA projects. An example of such a case would be the inability of a European company to store private data of European Union citizens on servers that are hosted in countries outside the EU that do not offer data privacy laws with the same protection of individual rights.
- JIRA as an internal service: Some of our large customers offer JIRA internally as a service. They have a standard configuration that they then clone for each business unit that requests a JIRA system. Each system then has its own BU-specific administrators while the central IT organization maintains system administration competency.
- Spreading Risk / Downtime: Larger instances take longer to upgrade. Therefore, one possibility to reduce unavailability of JIRA services to all users is to split and thereby reduce the size of instances. That way, an upgrade downtime of eight hours could then be split and turned into two-hour downtimes of four JIRA systems with each downtime affecting less users.
- Scaling: Last but not least, as stated in the JIRA State of the Union 2012, is scalability. If a JIRA server hits its limit and other optimization measures have already been implemented, then one of Atlassian's recommended approach to further scale is federation.
As an example of federation, let's share with you how Atlassian deploys dedicated instances for various purposes:
|Name||Purpose||Main reason to run on dedicated instance||Example Application links|
|Public facing issue management instance that includes new feature requirements, change requests and bugs. The best feature here is the possibility for customers to vote on issues.||Public / Private||Linking between JAC issues to related development issues in JDOG.|
|SAC||External support instance - explain links between SAC and JAC. Issues are visible to the reporter (support requester) and Atlassian only due to the classified nature of included information.||External Customer requirement, Public/Private||Linking between SAC support cases with related JAC issues (visible to the support customer) and JDOG (visible to Atlassian only).|
|JDOG||JIRA teams' dogfooding server - validates your notion of having something on the cutting edge of features (for us, this is a very unique case since JIRA is our product, but you get the idea).||Access to new features||Linking back to JAC and SAC respectively.|
|EACJ||Our internal JIRA that runs our business process, from marketing to procurement and internal IT.||Public / Private|
Best Practices for managing multiple JIRA instances
Federation of JIRA instances is quite common among enterprise customers. Past research by Atlassian counted hundreds of customers that run three or more JIRA servers concurrently within their organization. As a result, customers with federated environments are not alone, but part of a larger community. Atlassian has therefore developed and is continuing to implement features that make management and interoperability of federated JIRA instances easier for customers.
The ability to have Atlassian products work together is one of the strongest features of JIRA, Confluence, and the entire Atlassian stack. One of the best examples of this is the possibility to dynamically list JIRA issues within a Confluence page. This same functionality also enables the integration of autonomous JIRA instances. The JIRA servers, thereby, do not need to run the exact same version as long as all versions contain the Application Links functionality. This makes interoperability in a heterogeneous landscape (e.g. see reasons to federate, "Access to new features") a lot easier.
In a nutshell, the goal of application links is to make inter-instance linking and integration of issues as easy as inter-project integration within the same instance! So after an application link between your JIRA instances is set up, the integration functionality described below becomes available:
- Unified activity steams
- Unified reporting dashboards
- Remote issue links
- Application navigator
- Project links
- JIRA Issue Copy
- Federated search
Unified activity streams
Activity streams not only show updates from the instance you are on but also from connected JIRA or other Atlassian product instances (e.g. Confluence or FishEye). For example, two companies working together on a project and linking up their JIRA instances could be updated with new status changes of work at their partner's JIRA through a unified activity stream.
See for more information.
Unified reporting dashboards
Gadgets that draw their data from another linked and trusted JIRA instance can be added to a JIRA dashboard. This enables a unified reporting overview of all relevant instances. At Atlassian, for example, we utilize information radiators to show real-time reporting across our internal and external JIRA instances. Next to the standard gadgets, you can also add additional gadgets. The applications are endless – from just one gadget that reports on a specific metrics of another project in another instance, to having one central instance in your federation that runs reporting of all projects in all JIRA systems!
Remote issue links
Linking issues is crucial to connecting relevant JIRA issues. With remote links, you can not only track dependencies and other relations of issues to other local projects, but to remote projects in other instances as well. See about how we connect issues in JAC (JIRA.Atlassian.com) with internal development tickets in our JDOG instance.
You do not have to remember the addresses of your Atlassian products any more. The Application Navigator header allows for fast switching between JIRA instances, as well as other Atlassian products working together.
In many cases, you will have a project in an internal JIRA instance for the development of a certain product and then a project in an external instance for the support of that product. With project links, you can make a connection between these two projects easier. For example, an activity stream in JIRA1 that points to JIRA2 will automatically show related updates of a connected project.
Atlassian Labs, not supported)JIRA Issue Copy (by
Atlassian Labs has developed the JIRA to JIRA Issue Copy, which allows users to copy an issue from one JIRA instance to another. It is currently still in Atlassian Labs and available on the Atlassian Marketplace. It has a cleaner and easier to understand user interface, and supports more custom configurations and enhancements in the user matching algorithm. Note that this add-on is not supported by Atlassian, and although it may work for you now, we don't recommend building it into a process that you rely on due to it's Atlassian Labs status.
Although searching across multiple instances is not part of an out-of-the-box JIRA installation, you can achieve it through ALM's JIRA Client. This plugin is a desktop application that was designed to enable offline work with JIRA. It lets you establish connections to an unlimited amount of JIRA instances and once this connection has been established, search queries are executed automatically across all the various systems.
Administration of Federated Instances
The following sections describes JIRA features that you can use to efficiently administer a federated JIRA instance:
- Central user management
- Permission scheme harmonization
- Workflow sharing
- Migrate, merge and split
- Configuration management
- Synchronization of projects across instances
- Upgrading multiple instances
Central user management
JIRA offers multiple ways to have a single-sourced user management, group definition and authentication (SSO) that multiple JIRA instances or even other Atlassian products can draw from:
- Crowd: Atlassian's dedicated product to provide centralized identity management with single sign-on for an organization's applications and developer tools. See the chapter on Connecting to Crowd Server for User Management for more instructions. In case you want to migrate user directories from JIRA to Crowd, see Importing Users from JIRA to Crowd.
- LDAP: Is the de facto standard on storing and managing distributed directory information. See the JIRA documentation on how to connect to an LDAP Directory.
For an overview of different scenarios on how multiple JIRA systems and/or other Atlassian products could interoperate and accomplish centralized user management, see illustrated Diagrams of Possible Configurations for User Management.
Permission scheme harmonization
The ability to have a single source for user groups also facilitates the harmonization of permission schemes across multiple systems. Groups such as Project Administrators, System Administrators, Internal, External, etc. can be centralized and then assigned to JIRA's local permission schemes.
The Workflow Sharing feature allows you to share your team's workflow with other teams in your organization on different JIRA instances, or external parties in other organizations via the Atlassian Marketplace. This feature allows you to easily use workflows that other people have published, or to move a workflow from staging to production in your own organization. It is also provided as a plugin for older JIRA releases.
Migrate, merge and split
In a federated environment, multiple projects run on multiple instances of JIRA. In certain circumstances, it might be desirable to change the federation landscape by altering the number of instances involved or moving projects between them. This can be done by merging and splitting instances as well as through project migration.
Test first! For all the procedures below, we highly recommend performing this procedure on a staging server first before production systems are touched.
- Migrate a project to another instance - CSV import: If you only want to migrate one project, a quick way is to use the CSV Importer:
- Create the project in the target instance.
- Export the issues in the source project to a CSV Format.
- Import it into the target system using the CSV importer and use the wizard to match meta-data to the taret instance configuration.
Please note that a CSV import will only migrate the issue data itself. Plugin data stored in active objects will not be carried over.
For more information, please see Importing Data from CSV.
- Merge / Migrate - project import: One way to migrate a single project is to use the export function in the source system and then use project import in the target JIRA instance.
- Make sure that the two instances run the same versions of the JIRA and any plugins.
- Also ensure that the custom fields, user groups, entities like priorities and statuses as well as schemes related to the projects in the source system exist in the target system with the same nomenclature.
- Perform a full or partial XML export from the source system.
- Import the XML files into the target system.
An XML export of a project doesn't export all plugin specific active objects (such as Agile ranking). See JRA-28748 (Project Imports should include GreenHopper data)
- Splitting an instance - full XML backup restoration: Using the restoration feature, it is possible to clone a JIRA instance through a backup/restore procedure.
- Set up a new JIRA server with the same version of the existing system. This will require a new JIRA license.
- Perform a full backup of the source system.
- Import the backup into the new JIRA instance.
- Choose which duplicate projects to remove from the source and target instances.
- Please note that this option only applies if the target instance is empty.
For more detailed instructions, see Splitting JIRA applications.
Although it is possible to run multiple JIRA instances on the same server, dedicated hardware / VM is recommendable for enterprise use of JIRA.
- Splitting an instance - DB / Filesystem level: Some plugins may use the JIRA database in a way that can not be exported through the standard XML export.
- Set up another JIRA server and replicate the application server, database and installation / home directories. For detailed instructions, please refer to the backup chapter of the staging server guide.
- The steps here are essentially the same as the above: Splitting an instance - full XML backup restoration, however this time use your native database backup tools.
The more JIRA instances a federated system has the more there is a need for central administration of these systems, we see clients achieving configuration management by implementing various methods:
- Version Control System: A VCS like Subversion or GIT can be used to keep track of changes to files such as configuration XMLs or templates and can then be deployed to a staging or production system respectively. This approach provides the ability to roll-back changes and having one central overview of all configuration files in all instances.
- Configuration Management Tool: Many of our customers use configuration management software to keep track of their IT systems, versions, etc. An example would be Puppet in combination with a puppet-JIRA plugin.
- Continuous Deployment: CI applications like Bamboo can be used to automatically or manually deploy configuration changes to a JIRA's staging / production system.
- JIRA Workflow Sharing Plugin, by Atlassian Labs (not supported). Also mentioned above in the 'Merge/Split' section. Can be used to centrally maintain and configure workflows and share them across multiple JIRA systems.
Although designed to import content from 3rd party issue tracking systems, the Enterprise Migration Utility can also be used to migrate JIRA projects between instances or conduct merging operations.
The JIRA Auditor add-on maintains an audit log on configuration changes in JIRA. This is especially important when multiple administrators make changes to the same instance.
Synchronization of projects across instances
In some scenarios,when JIRA systems are integrated and deal with similar entities, it might be desirable to have actions in one system affect another. This is not an out-of-the-box feature but can be achieved through the use of plugins and scripting:
- Backbone Issue Sync for JIRA - supported by K15t Software GmbH
The Backbone Issue Sync for JIRA synchronizes issue data between different JIRA instances. Synchronize JIRA project data across departmental and B2B borders with ease, flexibility, and confidence.
- JJUPIN - Simple Issue Language (SIL) for JIRA, supported by Kepler-Rominfo
The JJUPIN plugin provides a new scripting layer for JIRA that enables administrators to create new functionality. It is discussed in the marketplace as being a solution for synchronization and cross-system workflows for federated JIRA environments.
- JIRA Enterprise Mail Handler, supported by Javahollic Software
The JIRA Enterprise Mail Handler is a plugin which was originally designed to provide advanced functionality to handle incoming emails. This includes the interaction with users that do not have a JIRA account or users that use several email aliases. Through its project and field mapping, it can also be used to synchronize and update issues between different instances. For example, you can automatically create an issue in instance A if the issue transitions to a particular state in instance B. Setting up such automation however should be thoroughly tested as there can be occurrences of infinite loop if issues are updated at the same time on both instances.
- ScriptRunner, supported by Adaptavist
Further customizations are also possible by creating Jelly Scripts that use the REST API of federated JIRA systems. Also, a custom plugin could be developed to provide such functionality.
Upgrading multiple instances
In order to avoid manual repetition of the upgrade instructions given in the JIRA administration manual, there are ways to partially or fully automate the process:
- Automated Backup: You can export data from JIRA using the JIRA Command line interface. (The JIRA CLI can also be quite useful for automated tests).
- Symlinks: Instead of working only with the installation directory, a system administrator could maintain a directory for each JIRA version. Once the new version has been installed and modified, a symbolic link could be modified to point to the folder of the new version. This is a practice that has proven itself at Atlassian internally.
- Automating JIRA operations: It is possible to automate operations in JIRA via wget. Make sure to test such scripts with each new version of JIRA as such automation relies on the web interface of JIRA which might change from version to version.
For JIRA instances with high availability requirements, it is also recommended to use a staging server setup for JIRA. You do not require a separate license for a staging server.
As discussed in the Application Links section, JIRA instances don't have to run the same version in order to enable application links.
Atlassian's partner Appfire offers an Upgrade Assistant for JIRA that automates many procedures around the updating process out-of-the-box.
Planning HA / failover for multiple instances
Federating JIRA instances will enable you to set different availability requirements for your instances. E.g. some JIRA servers might run vital processes that require an uptime only achieved with failover mechanisms while others are not mission critical at all. You can certainly leverage the same tools for many instances such as the load balancer (given it runs in HA mode as well) for multiple JIRA instances.
Please refer to the Failover for JIRA Guide for more information.
Some of the operations described in this guide such as a customized synchronization between JIRA systems require very sophisticated knowledge of scripting and system administration. We recommend collaborating with one of our Experts for such undertakings.
We can help connect you with a right Partner / Expert. If you're interested in a referral, please contact us.
Please also contribute your best practices, questions and comments around federating JIRA to Atlassian Answers.