Cloud
Data Center 9.15
Versions

Jira Software guardrails

Performance and scaling

On this page

Related content

  • No related content found

Still need help?

The Atlassian Community is here for you.

Ask the community

The content of this page applies to Jira 9.14. If you're looking for information about a different version, select it from the menu in the top-right corner.

Background

We’re committed to supporting the needs of our largest customers, and this includes continually improving the performance and scalability of our products. The amount of data in your instance can be a factor in performance and stability problems. As your instance grows, so does your risk of performance degradation over time. Often this is a gradual degradation and can go unnoticed until you reach a point where it has a significant impact on your team.

In the table below, we’ve described the performance and stability impacts that we’ve observed and suggested some actions you can take to reduce your risk. The guardrails are based on real-world experiences with some of our largest customers, but won’t necessarily be representative of every organization’s experience.

Ways you can reduce the risk of experiencing serious performance and stability problems may include:

  • application changes, such as upgrading to a newer application version to get the benefit of performance improvements, or changing the way users are managed.

  • infrastructure changes, such as increasing memory, CPU, or running a cluster or mirrors.

  • data cleanup activities to reduce your footprint, such as archiving or breaking up monolith sites.

It’s important to note that these aren’t hard limits, and some of your product instances may already exceed these thresholds. There are a number of factors, including the interplay between different data types, and site load, which will influence whether you experience the potential impacts listed below, and to what degree. As with any type of risk, it’s essential to identify the risk and make a plan, so you can prioritize those actions that will help you reduce the probability of future performance problems.

Definition

Product Guardrails are data type recommendations designed to help you identify potential risks and aid you making decisions about next steps in your instance optimization journey.

Jira Software guardrails

The following guardrails are provided to help you identify and mitigate scale risks, and make decisions about cleaning up your instance.

Projects

Content type

Number of active projects

Guardrail

7000 projects (not archived)

How to find this number

How to identify old projects to clean up

Risks

We've observed these problems when operating above this guardrail:

  • Number of projects makes permission calculation complex and therefore slower.

  • When creating a new project the application 'hangs' for 20-50 seconds per active node in the cluster (observed with 10k projects), and displays a timeout error after 60 seconds, even if the project is created successfully.

  • Reindexing takes a long time.

Mitigation options

Comments

Content type

Number of comments per issue

Guardrail

1000 comments per issue

How to find this number

How to find issues with the most comments in the database

Risks

We've observed these problems when operating above this guardrail:

  • Issue view loads slowly.

  • Out of memory errors, which can lead to application crashes (in extreme cases).

  • Reindexing takes a long time.

Mitigation options

  • Moderate a user group’s activity by setting a global limit on the number of specific items its members can create with Safeguards

  • Remove older comments via REST or using ScriptRunner.

  • Check any automation to make sure you're not adding unnecessary comments. Consider reducing the frequency or batching updates.

  • Implement rate limiting to prevent a misconfigured integration from adding thousands of comments in a short space of time. Learn about rate limiting

Attachments

Content type

Number of attachments per issue

Guardrail

3000 attachments per issue

10MB per single attachment

How to find this number

How to find issues with the most attachments in the database

Risks

We've observed these problems when operating above this guardrail:

  • Issue view loads slowly.

  • Load on shared home filesystem (for example, while loading of thumbnails)

  • Issue operations (such as view, update) or post-processing can trigger serialising the issue with all the issue properties, including all issue attachment properties.

Mitigation options

Content type

Number of issue links or sub-issues

Guardrail

1000 issue links

How to find this number

How to find issues with the most issue links in the database

Risks

We've observed these problems when operating above this guardrail:

  • Issue view loads slowly.

  • Can results in stuck threads, which can affect the whole instance.

Mitigation options

  • Identify issues with many issue links, and remove any unnecessary links

  • Archive issues that are no longer needed. Learn how to archive issues

Text

Content type

Amount of text in a field.

Guardrail

255 characters in single-line fields

100k characters in description and multi-line fields

How to find this number

Configuring advanced settings

Risks

We've observed these problems when operating above this guardrail:

  • Increased index size.

  • Slow search results.

Mitigation options

Custom fields

Content type

Total number of custom fields

Guardrail

1,200 custom fields

How to find this number

Analyze the usage of custom fields

Risks

We've observed these problems when operating above this guardrail:

  • Overall performance degradation.

  • Reindexing takes a long time.

Mitigation options

Epics

Content type

Number of epics

Guardrail

120,000 epics

How to find this number

You can use JQL to identify the number of epics, issuetype = Epic

Risks

We've observed these problems when operating above this guardrail:

  • Epic link menu loads slowly.

Mitigation options

Sprints

Content type

Total number of sprints

Guardrail

60,000 sprints

How to find this number

How to find the total number of sprints in the database

Risks

We've observed these problems when operating above this guardrail:

  • Overall performance degradation due to slow sprint cache population.

  • closedSprints() JQL function does not work (limited to 65,000 sprints).

Mitigation options

Workflow scheme bulk actions

Action

Associating a new issue type to an existing workflow scheme

Guardrail

1000 issues per bulk action

How to find this number


Risks

We've observed these problems when operating above this guardrail:

  • Bulk action can take a very long time to complete (several days)

  • Can’t view progress of a workflow scheme modification without shortening the URL

Mitigation options

  • Copy the original workflow scheme, make the change, then associate the workflow scheme project by project.

  • Do nothing. The background process will take a long time to complete, but it’s not resource-intensive and won’t cause performance issues. Make sure you don’t restart Jira until it has finished.

Change history

Content type

Number of changeitems or changegroups associated with an issue

Guardrail

20,000 changeitems or changegroups

How to find this number

Retrieve issue change history

Risks

We've observed these problems when operating above this guardrail:

  • Out of memory errors when viewing the History tab.

  • Issue view and other issue actions load slowly.

  • Reindexing takes a long time.

Mitigation options

  • Use a database query to identify issues with large changeitems and changegroups, then clone the issue, as the history is not copied to the new issue.

Users

Content type

Total number of users synchronized between LDAP and Jira

Guardrail

100,000 users

How to find this number

How to get the total number of users

Risks

We've observed these problems when operating above this guardrail:

  • Instance instability including outages and noticeable performance drops under heavy load
  • Increased time for directory synchronization and user authentication

Mitigation options

  • If most of the user accounts in your instance are stored in Crowd Data Center or Microsoft Active Directory, enable incremental synchronization. This way, only the changes since the last synchronization will be queried, reducing the need for a full sync. For more information, see Connecting to an LDAP directory.
  • Consider using Crowd Data Center as your external user directory to take advantage of features such as access-based synchronization. For more information, see Syncing users based on their access rights
  • Use LDAP filters to reduce the number of users and groups to process by your instance. For more information, see:
    • Connecting to an LDAP directory
    • Reducing the number of users synchronized from LDAP to JIRA applications 
    • How to write LDAP search filters  
  • Become familiar with User management limitations and recommendations.

Groups

Content typeTotal number of groups synchronized between LDAP and Jira

Guardrail

25,000 groups
How to find this numberHow to get the total number of groups
Risks

We've observed these problems when operating above this guardrail:

  • Instance instability including outages and noticeable performance drops under heavy load
  • Increased time for directory synchronization and user authentication
  • Application access and group management UI unresponsiveness
Mitigation options
  • Configure your LDAP connection pool. Too many or too few connections may have a negative impact on performance. For more information, see Configuring LDAP connection pooling.

  • Disable group sync on every login by changing the Update group membership when logging in option to For newly added users only or Never. For more information, see Connecting to an LDAP directory.

    Changing this setting means that group membership data will not be updated until the next directory synchronization.

    Advances Settings section on the user directory configuration page in Jira

  • Become familiar with user management limitations and recommendations.


Depth of nested groups

Content type

Number of levels of hierarchy when groups are nested

Guardrail

4 levels deep

We also recommend groups do not contain a mix of users and other groups, as this can have a negative impact on performance.

How to find this numberHow to check the depth of group nesting
Risks

We've observed these problems when operating above this guardrail:

  • Instance instability including outages and noticeable performance drops under heavy load
  • Increased time for directory synchronization and user authentication
Mitigation options

Try rebuilding your group structure to prevent deep nesting. For example, you can split your group structure into two categories:

  • groups containing only user accounts (and not other groups)
  • groups containing only other groups (and not individual accounts)

Nested groups come with their own set of limitations and potential side effects. Make sure that you understand this mechanism before rebuilding your group structure. For more information, see Managing nested groups.

Show me an example...

For a better understanding of what this type of structure might look like, imagine the following simplified scenario, where an organization defines some high-level groups:

  • staff for all of the organization’s employees
  • engineering for members of the engineering department
  • design for members of the design department
  • marketing for members of the marketing department

In this example, we’ll focus on the engineering group. The group is part of the larger staff group and contains only smaller sub-groups representing separate Scrum teams (and not their members' accounts); for example dev-a and dev-b. The staff group does not store any user accounts itself, only the sub-groups for each department in the company.

By making sure that individual accounts are added only to the dev-a and dev-b sub-groups of engineering, you’ve reduced the level of nesting to a maximum of three while keeping an easy-to-maintain permission inheritance scheme.

The following tree diagram illustrates this hierarchy:

staff/
├─ engineering/
│  ├─ dev-a/
│  │  ├─ jsmith@acme.com
│  │  ├─ jdoe@acme.com
│  │  ├─ mdavis@acme.com
│  ├─ dev-b/
│  │  ├─ rlewis@acme.com
│  │  ├─ nphillips@acme.com
│  │  ├─ tadams@acme.com
├─ design/
├─ marketing/

Automation for Jira

Automation for Jira guardrails is the derivative of the general health of the instance. When assessing the performance of rule executions, keep in mind the performance and scaling guardrails for Jira Software and Jira Service Management. 

Learn more about the best practices for optimizing automation rules

Create/Update/Delete rule

Content typeHow often rules are created/updated/deleted
Guardrail

Instance with:

  • 1–5000 rules → one edit per 1 minute
  • 5001–10.000 rules → one edit per 5 minutes
  • 10.001–15.000 rules → one edit per 15 minutes
  • 15.000+ → not recommended
How to find this number
(warning) Requires access to the database

Replace the X with the corresponding minute value. Adjust the singular/plural ending of the time unit in a query (only for PostgreSQL).

SELECT "UPDATED"
FROM "AO_589059_RULE_CONFIG"
WHERE "UPDATED" >= NOW() - INTERVAL 'X MINUTE/S';
SELECT UPDATED 
FROM AO_589059_RULE_CONFIG
WHERE UPDATED >= NOW() - INTERVAL X MINUTE;
SELECT UPDATED 
FROM AO_589059_RULE_CONFIG
WHERE UPDATED >= DATEADD(MINUTE, -X, GETDATE());
SELECT UPDATED
FROM AO_589059_RULE_CONFIG
WHERE UPDATED >= SYSDATE - INTERVAL 'X' MINUTE;
Risks

We've observed these problems when operating above this guardrail:

  • High frequency of create/update/delete operations on the rule configurations will trigger an expensive EventRuleCache refresh.

  • Issue-related actions (creating, editing, deleting) are delayed 20-50 seconds because they wait until the EventRulesCache loads after applying changes.

  • HTTP threads responsible for consecutive automation rule configuration changes and issue operations are blocking:

    • http-nio-8080-exec-x URL: /jira/rest/cb-automation/latest/project/GLOBAL/rule/xxxx (…) parking

    • http-nio-8080-exec-x URL: /jira/rest/api/2/issue/ (…) parking

  • HTTP threads responsible for rule executions are waiting (automation-rule-executor:thread-x parking).

Mitigation options

  • Reduce the number of automation rules (the cache will load faster).

  • Perform fewer create/update/delete operations on automation rules (rule configurations).

  • Review why users need to update rules so often.

  • Reduce the number of users with rule managing permissions.

  • Schedule a dedicated time for rule administration purposes and communicate internally. It will make the users aware of the possible delays.

Automation rule executions

Content typeThe number of daily rule executions
Guardrail10,000,000
How to find this number

(warning) Requires access to the database

SELECT COUNT(*)
FROM "AO_589059_RULE_STAT"
WHERE "CREATED"::DATE = NOW()::DATE;
SELECT COUNT(*)
FROM AO_589059_RULE_STAT
WHERE DATE(CREATED) = CURDATE();
SELECT COUNT(*)
FROM AO_589059_RULE_STAT
WHERE CAST(CREATED AS DATE) = CAST(GETDATE() AS DATE);
SELECT COUNT(*)
FROM AO_589059_RULE_STAT
WHERE TRUNC(CREATED) = TRUNC(SYSDATE);

From the UI (aggregated per rule)

To find a number of daily rule executions:

  1. Go to Administration > System > Automation rules (under Automation for Jira).

  2. Select the more actions menu … > View performance insights > 1d (one day), and then from the dropdown menu under the graph, select Execution count.

Risks

We've observed these problems when operating above this guardrail:

  • Disk space ends because of the high utilization of TempDB/Version store for AO_589059_RULE_STAT_ROLLUP_DAY.
Mitigation options

Learn more about Jira Data Center automation release notes

Find out more about Automation for Jira version compatibility:


Rules with JQL

If the rule contains the JQL search we should assess if it won’t cause performance issues.

Content typeThe complexity of JQL queries
GuardrailMust be assessed individually (per rule)
How to find this number

App Diagnostics 

JQL monitors are available under
 <JIRA_URL>/plugins/servlet/diagnostics/overview.

Learn more about Jira Data Center monitoring

Slow JQL log

Records JQL queries and their sources for executions exceeding the default threshold of 400ms: atlassian-jira-slow-queries.log

Threads related to Automation for Jira will have the following prefix: automation-rule-executor:thread-#

Learn more about useful Jira log files

Risks

We've observed these problems when operating above this guardrail:

  • Slow execution of rules.

  • Blocking automation threads.

  • Depending on the JQL case, slower performance of the affected parts of Jira.

  • Not responsive instance as the worst-case scenario.

Mitigation options

Automation for Jira monitoring

Automation for Jira has a set of monitoring tools available.

Learn more about how to monitor automation activity and diagnose issues

For large-scale instances, it’s highly recommended to:

Last modified on Mar 25, 2024

Was this helpful?

Yes
No
Provide feedback about this article

Related content

  • No related content found
Powered by Confluence and Scroll Viewport.