Managing custom fields in JIRA effectively

Overview

The ability to set custom fields makes Jira a powerful workflow management tool, allowing you to create highly specific objects for modeling the way your organizations work. However, like all good things, too much can be detrimental to Jira's performance and administration. Here, we explain how using custom fields affects Jira's performance; we also provide several recommendations on its proper use and optimization.

Too many custom fields can slow down your instance

The response time of any action that requests or processes custom issue details (viewing, searching, creating/editing issues, and adding comments) is affected by the raw number of custom fields in Jira. Our scaling tests for Jira 7.7 revealed as much: 

Figure 1: Average response times per action (in milliseconds): custom fields test

When we tested 1 different user actions on Jira instances with 1400 and 2800 custom fields, the latter showed significant differences in response times for Add CommentCreate Issue, and Edit Issue. Considering that users spend most of their time on Jira performing these three actions, it's clear that you need to pay close attention to the number of custom fields in your Jira instance. 

We also tested the same set of user actions across eight other Jira instances, doubling a different data point each time. The following table shows aggregates of all average response times, for all actions in each test:


Figure 2: Aggregates of average response times (in milliseconds) for all actions: custom fields test

This graph suggests that of the eight data points we tested, the number of custom fields had the greatest effect on average response times. 

Identify and clean up unused custom fields

Here's three strategies for reducing custom field bloat in your instance:

  1. Delete any unused custom fields. 
  2. Merge custom fields that see very little use.
  3. Limit the scope of the custom fields to specific projects context. 

Before removing or consolidating any custom fields, discuss them with Business Owners and ensure that your changes are tested in a QA environment before rolling them out to production. This is a time-consuming process, but you need to be careful so that no critical data is lost. 

tip/resting Created with Sketch.

Limiting custom field scope

For any custom fields that can't be removed or merged safely, you can limit their scope to the context of specific projects. This helps by:

  • Reducing the index size (Jira Data Center search indexing),
  • Lowering computation complexity, and
  • Reducing the work done by other nodes in the cluster during replication.


SQL Database queries

You can use the following SQL queries to retrieve the information you need and take appropriate action:


Unused custom fields

This SQL query can also return custom fields implemented by plugins; as such, those custom fields may be using their own datastore. Have these custom fields reviewed by Business Owners before taking any action.

select customfield.id
from customfield left join customfieldvalue on customfield.id = customfieldvalue.customfield
where customfieldvalue.stringvalue is null and customfieldvalue.numbervalue is null and customfieldvalue.textvalue is null
order by customfield.id
Custom fields with low usage

This query returns all custom fields that have been use less than five times.

select customfield.id, count (*)
from customfield left join customfieldvalue on customfield.id = customfieldvalue.customfield
group by customfield.id having count (*) < 5
order by count (*) desc
Custom fields that have not been updated after date (YYYY-MM-DD)

This query also excludes Agile, Service Desk, Capture, and Portfolio fields.

select field.id, field.cfname from customfield field where field.cfname not in (
select item.field from changeitem item
JOIN changegroup cgroup ON item.groupid=cgroup.id
where item.fieldtype='custom' and cgroup.created > 'YYYY-MM-DD'
) and customfieldtypekey not like '%com.pyxis.greenhopper%'
and customfieldtypekey not like '%com.atlassian.servicedesk%'
and customfieldtypekey not like '%com.atlassian.bonfire%'
and customfieldtypekey not like '%com.atlassian.jpo%'
User activity related to a custom field

This query uses the complete custom field ID (for example, customfield_10301).

select id, entitytype, entityid, username, to_timestamp(lastviewed/1000) as lastviewed, data from userhistoryitem 
where entityid='customfield_10001'
order by lastviewed desc;
All 'Select list' custom fields
select * from customfield 
where customfieldtypekey = 'com.atlassian.jira.plugin.system.customfieldtypes:select';
Filters which contain a "Custom Field Name" or custom field id "10001"
select * from searchrequest where reqcontent like '%Field Name%' or reqcontent like '%10001%';
Workflows that reference a custom field

This query uses the complete custom field ID (for example, customfield_10301).

select * from jiraworkflows wf where wf.descriptor like '%customfield_10301%';
Custom fields with global context

Applying a global context to a custom field makes it available on all projects.

SELECT distinct(p.pkey),
cfname,
cf.id,
p.pname                          
FROM CUSTOMFIELDVALUE cfv
left join JIRAISSUE ji
on cfv.ISSUE = ji.id            
right join CUSTOMFIELD cf
on cf.id = cfv.customfield
left join project p
on p.id = ji.project
WHERE cf.id in (''' + globalContextIdsCommaSeparated+ ''')
GROUP BY cf.id,cf.cfname,p.pkey, p.pname
ORDER BY cf.cfname,p.pname
Non-calculated fields with contexts & no values, which do not have Screen/Workflow/Notification schemes
SELECT
cf.id id,           
to_char(min(ji.created), 'YYYY/MM/DD'),
to_char(max(ji.updated), 'YYYY/MM/DD'),
count(distinct(ji.id))
FROM CUSTOMFIELDVALUE cfv
left join JIRAISSUE ji
on cfv.ISSUE = ji.id            
left join CUSTOMFIELD cf
on cf.id = cfv.customfield            
GROUP BY cf.id
ORDER BY cf.id
Non-calculated fields with contexts, no values, and do not have Workflow/Notification scheme, but is mapped to some screens
SELECT
cf.id id,           
to_char(min(ji.created), 'YYYY/MM/DD'),
to_char(max(ji.updated), 'YYYY/MM/DD'),
count(distinct(ji.id))
FROM CUSTOMFIELDVALUE cfv
left join JIRAISSUE ji
on cfv.ISSUE = ji.id            
left join CUSTOMFIELD cf
on cf.id = cfv.customfield            
GROUP BY cf.id
ORDER BY cf.id

Context discovery

You can also use the following script to determine the context of each custom field:

https://bitbucket.org/morzechowski/context-discovery-for-custom-fields

This script returns a list of all custom fields with projects that have at least one issue with a value within the field. It includes a filter that removes all fields that are used in more than 50% of all projects to exclude most commonly used fields.

This script does not alter anything in the database; rather, it just lists fields and projects.

Unsupported solution

This script isn't officially supported, and it's only listed here to provide additional insight (if needed). We highly recommend that you test this in a non-production environment.

Atlassian Marketplace plugins to identify/remove custom fields

Set up governance around custom fields

Once unused or rarely used custom fields have been identified and removed successfully from an instance (or merged), set up rules and guidelines for creating new custom fields in the instance. This will help establish a disciplined approach, helping prevent it from affecting your instance's performance. Establishing governance around Jira customization is useful in balancing the growth of the instance with its ability to serve the needs of its users.

A few things to keep in mind here:

Evaluate requests for new custom fields: When users request new custom fields, establish a process for vetting the necessity of the request. Meet with the users to interpret their requirements for the fields and figure out if an existing field can be repurposed or if a field is needed at all. 

Verify the purpose of new custom fields: Ensure that the information captured by each field serves a justifiable purpose. If the data is not used to work on the issue, or for reporting, then it is not needed. Track the number of custom fields actually being used on a quarterly basis or as frequently as it is practical in your organization.

Establish and share a governance policy: Create a clear and definitive set of regulations that establish how new objects such as custom fields, workflows, and other customizations are added to Jira. 

Get executive backing Share your governance policy with relevant managers and executives at your organization and emphasize the need for their support in keeping Jira running smoothly. Be ready with facts to substantiate your policies and show how the lack of governance affects the performance of the instance and ultimately impact the end-users. You can even show the test results from this article.

Other recommendations

Some other useful Do's and Don'ts for managing custom fields  are summarized below:

DO

      • Reuse existing fields when possible.
      • Make field names as generic as possible.
      • Give your custom fields useful descriptions, including examples of valid values.
      • Format custom fields consistently (for example, capitalization). Whenever possible, keep them consistent with Jira defaults.
      • Be careful of trailing spaces and other non-visible characters added to custom field names. These will make using scripts harder.
      • Err on the side of flexibility. For example, a multi-select field is more flexible than a single-select field.
      • Translate custom field names for your users who are using a different language pack.
      • Know how and when to use custom field contexts.
      • Use the Where is my field? feature to track down missing fields.
      • Have a governance policy and make it easy for users and executives to find.
      • Use custom fields contexts appropriately. For example, avoid using global context when a custom field is only being in used in one project. 
      • Avoid setting default values for your custom fields whenever possible.
      • When creating custom fields with a drop-down list, try to minimize the number of options available. 

DON’T

      • Don’t create custom fields with duplicate names. This can lead to administration mistakes, reporting issues, and custom script errors.
      • Don’t create option fields that have to be updated frequently. For example, a select list with usernames will quickly become outdated as people’s roles change. 
      • Don’t add the option value None. This option already exists when the field is made option and adding it to the list makes reporting harder.


References

You can also refer to Practical JIRA Administration: Using JIRA Effectively: Beyond the Documentation by Matt Doar for more advice on effectively managing custom fields.

The data points used in this article were collected as part of a series of scaling tests between different versions of Jira. For more details about the hardware, results, and methodology of those tests, see Scaling Jira 7.7.

We're here to help

This article features advice typically provided by Atlassian Technical Account Managers to customers when custom field use starts affecting performance. Technical Account Managers can provide strategic guidance to help you choose the right methods to manage your custom fields or even mitigate other performance problems.

Our Premier Support team performs health checks by meticulously analyzing your application and logs, to ensure that your application's deployment fully meets the needs of your users. If the health check process reveals any performance gaps, Premier Support can recommend possible changes to your instance.

Last modified on Apr 27, 2018

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.