The way a search is performed in JIRA has significantly changed. The introduction of advanced searching (JQL) necessitated a rewrite of the JIRA searching subsystem. In the process, the API for searching has also been changed (and improved) significantly. Unfortunately these changes will almost certainly mean that plugins that search will need to be updated for JIRA 4.0.
In JIRA 3.x and earlier, searching was achieved using a
SearchRequest in combination with
SearchSorts. While the
SearchRequest still continues to exist in JIRA 4.0, the
SearchParameters have been replaced with the
Query is JIRA's internal representation of a JQL search. It contains the search condition (i.e. the "where" clause) and the search order (i.e. the "order by" clause). The
Query object can be created using the
JqlQueryBuilder. For example, to create a query "find all issues assigned to either Dylan or Tokes that are unresolved and due in the next week" you would call:
Query has been obtained, it can be used to execute a search. In JIRA 4.0 a new
SearchService has been added to provide a central location for
Query related operations. To run the search you can simply call
SearchService.search() as documented on the
SearchProvider is still available for those who need to control the finer details of searching.
Query object is immutable; once it is created it cannot be changed. The
JqlQueryBuilder represents the mutable version of a
Query object. The
JqlQueryBuilder can be primed with an already existing
Query by calling
In JIRA 3.x the
SearchRequest was the object that was passed to the searching system to perform a search. The
Query object has taken over this role in JIRA 4.0; the
SearchService now take in
Query objects rather than
SearchRequest object has been reworked in JIRA 4.0 to significantly reduce its responsibility. For instance, ordering information is now stored on the
Query object rather than on the
SearchRequest object. The
SearchRequest really represents a saved search (aka. filter). You should only need to deal with
SearchRequests if you are working with filters. Even in this case, all searching operations need to be performed on
Query objects by calling
It is often necessary to get a URL for a particular
SearchService provides the
getQueryString(query) method for this. The method returns a parameter snippet of the form
jqlQuery=<jqlUrlEncodedQuery>, which can be appended safely to an existing URL that points at the Issue Navigator. Note that the links that JIRA 4.0 generates are JQL based, so are incompatible with JIRA 3.x and before. Old valid JIRA 3.x URLs will still work with JIRA 4.0.
Query object it is possible to retrieve its JQL representation by calling either
getJqlString(query) on the SearchService. The service makes sure that any values in the
Query that need to be escaped are handled correctly. Importantly, the
Query.toString() method does not return valid JQL (on purpose).
SearchService.parseQuery(jqlString) method can be used to turn a JQL string into its
Query representation. The return from this method has details on any parse errors encountered.
Query object, especially those parsed directly from the user, may not be valid. For example, the user may be trying to find issues in a status that does not exist. The
SearchService.validateQuery(query) method can be used to see if a particular
Query object is valid. Errors are returned with messages that can be displayed to the user. Executing an invalid
Query will not result in any errors and in fact may return results. To run an invalid query, JIRA will just make the invalid conditions equate to false and run the query. For example, searching for
status = "I don't Exist" or user = bbain will result in the query
<false> or user = bbain actually being run.
There are some methods on the
SearchService that we did not discuss here. Check out documentation on the
SearchService for more information.
Here's a complete example how to obtain search results for the query "project is JRA and the reporter is the currently logged in user and custom field with id 10490 contains 'xss'":
The preceding search could have also been written using the QueryBuilder:
Plugging into JQL and what happened to my Custom Field Searchers
The introduction of advanced searching (JQL) necessitated a rewrite of the JIRA searching subsystem. Unfortunately these changes will certainly mean that any
CustomFieldSearchers will need to be updated to work in 4.0.
The most fundamental change is that all JIRA 4.0 searching is implemented using JQL. A JQL search consists of two components: firstly, a number of conditions, or
Clauses, that must be matched for an issue to be returned; and secondly, a collection of search orderings that define the order in which the issues should be returned. The
Query object is JIRA's internal representation of a search. It is now the responsibility of the
CustomFieldSearcher to take a relevant
Query, validate its correctness and generate a Lucene query to find issues that match it. By doing this your custom field becomes searchable using JQL.
CustomFieldSearcher and/or the custom field is also responsible for ordering results if the order in the search includes the custom field. If your custom field ordered correctly in JIRA 3.x, then it will order correctly in JIRA 4.0. While the internal representation of an order has changed in JIRA 4.0, it still uses the same interfaces to order the search results. We will not address ordering again.
What is a JQL Clause?
A custom field must process the
Clauses from a JQL search to integrate into JQL. Each
Clause consists of a number of conditions (e.g.
abc != 20) combined by the AND and OR logical operators (e.g.
abc = 20 AND (jack < 20 OR jill > 34). In JIRA a condition is represented by a
TerminalClause, the logical AND by an
AndClause and a logical OR by an
OrClause, all of which implement the
Clause interface. Finally, the logical NOT operator can be used to negate any other
Clause. It is represented by a
NotClause that also implements Clause. These
Clause objects are composed together to represent a complex conditions. For example, the condition
abc = 20 AND NOT(jill > 34 OR NOT jack < 20) is represented by the following tree:
Clause can be navigated by passing an instance of a
ClauseVisitor to the
accept method of a Clause. This follows the traditional visitor pattern.
TerminalClause represents a
Clause of the form "field operator value". Inside the
TermincalClause the "operator" is one of the values from
Operator enumeration while the "value" is represented as an
Operand can represent a single value (e.g.
field = "single"), a list of values (e.g.
field in ("one", 1235)), a function (e.g.
field = function(arg1, arg2)) or EMPTY (e.g.
field is EMPTY). In the end, all you want is the values from the
Operand. These can be obtained as a list of
QueryLiteral (see below) by calling
JqlOperandResolver also has the
isValidOperand methods that can be used to determine the type of the
QueryLiteral represents either a
EMPTY value. These three represent JQL's distinguishable types. It is up to the
CustomFieldSearcher to convert these values into something that makes sense to it. The type of a
QueryLiteral can be determined by calling its
getStringValue methods. The get methods will return
false when the method and the
QueryLiteral type do not match.
Integrating with JQL
In JIRA 3.x a
CustomFieldSearcher was the way to provide customised searching functionality for custom fields. In JIRA 4.0 it is still the plugin point for searching; however, the
CustomFieldSearcher interface has changed significantly to accommodate the introduction of JQL. One of the major changes is that the
CustomFieldSearcher must return a
CustomFieldSearcherClauseHandler in JIRA 4.0. This object is a composition of a
ClauseValidator and a
ClauseValidator is used by JIRA to ensure that a JQL query is valid according to the
It is up to the validator to ensure that the operator and the value from the passed
TerminalClause makes sense for the
CustomFieldSearcher and its associated custom field. Any errors can be placed in the returned
MessageSet. They should be internationalised with respect to the passed user.
validate method must always return a
MessageSet as its result. A
null return is not allowed. A
MessageSet is an object that contains all of the errors and warnings that occur during validation. All messages in the
MessageSet need to be translated with respect to the passed searching user. An empty
MessageSet indicates that no errors have occurred. A
MessageSet with errors indicates that the JQL is invalid and should not be allowed to run. The returned messages will be displayed to the user so that any problems may be rectified. A
MessageSet with warnings indicates that the JQL may have problems but that it can still be run. Any warning messages will be displayed above the results.
ClauseValidator does not need to check if the passed
TerminalClause is meant for the for it, JIRA makes sure that it only passes
TerminalClauses that the
ClauseValidator is meant to process. It does that by only passing
TerminalClauses whose "field" matches one of the names the custom field must handle.
ClauseValidators need to respect JIRA security. A
ClauseValidator should not leak information about JIRA objects that the searcher does not have permission to use. For example, a
ClauseValidator should not differentiate between an object not existing and an object that the user has no permission to see. A
ClauseValidator that behaves badly will not cause JQL to expose issues that the searcher is not allowed to see (since JQL does permission checks when it runs the filter), though it does open up an attack vector for information disclosure.
ClauseValidator must be thread-safe and re-entrant to ensure correct behavior. JIRA will only create one instance of the
ClauseValidator per custom field instance. This means that multiple threads may be calling the validator at the same time.
ClauseQueryFactory is used by JIRA to generate the Lucene search for a JQL Clause.
It is the responsibility of the
ClauseQueryFactory to create the Lucene search for the passed
QueryCreationContext. The generated Lucene search is returned in the
QueryFactoryResult. The result contains the search (a Lucene Query object which is not related the the JQL Query object) and a flag to indicate whether or not the Lucene search should be negated. When set to true, JIRA will actually only match issues that do not match the returned Lucene search. For example, a
ClauseQueryFactory may decide to implement a condition like
field != value by returning a Lucene search that matches
field = value and setting the flag to true. You can also implement this condition by returning a Lucene search that matches
field != value and setting the flag to false.
The new argument here is the
QueryCreationContext. This object contains the variables that may be necessary when creating the query. The
QueryCreationContext.getUser method returns the user that is running the search and as such should be used to perform any security checks that may be necessary. The
QueryCreationContext.isSecurityOverriden method indicates whether or not this function should actually perform security checks. When it returns
true, the factory should assume that the searcher has permission to see everything in JIRA. When it returns
false, the factory should perform regular security checks.
ClauseQueryFactory should try to limit the queries so that issues that the user cannot see are excluded. Consider the query
affectsVersion = "1.0". The
ClauseQueryFactory might detect that there are two versions named "1.0", one from
project1 and the other from
project2. The factory might then notice that the user doing the search cannot see
project1. The factory can then return a query that contains only the version from
project2. This is mainly an efficiency concern as JIRA filters all search results to ensure users cannot see issues they are not allowed to.
ClauseQueryFactory does not need to check if the passed
ClauseQueryFactory is meant for it; JIRA makes sure that it only passes
TerminalClauses that the
ClauseQueryFactory is meant to process. It does that by only passing
TerminalClauses whose "field" matches one of the JQL names the custom field must handle. Put simply, the
ClauseQueryFactory must handle any passed
ClauseQueryFactory must also handle the situation when an invalid
TerminalClause is passed to it. An invalid
TerminalClause is one whose associated
ClauseValidator would not validate. The
ClauseQueryFactory must return an empty Lucene search if the passed
TerminalClause is invalid. Most importantly, the
ClauseQueryFactory must not throw an exception on an invalid
ClauseQueryFactory needs to be careful when implementing any of the negating operators (i.e. !=, !~, "not in"). These operators should not match what is considered empty by the custom field and
CustomFieldSearcher. For example, the JQL query
resolution is EMPTY will return all unresolved issues in JIRA. The query
resolution != fixed will only return all resolved issues that have not been resolved as "fixed", that is, it will not return any unresolved issues. The user has to enter the query
resolution != fixed or resolution is EMPTY to find all issues that are either unresolved or not resolved as "fixed".
ClauseQueryFactory also needs to consider field visibility. A
CustomFieldSearcher should not match any issues where its associated custom field is not visible. Importantly, asking for
EMPTY should not match issues where the custom field is not visible. For example, the JQL query
resolution is EMPTY will not return issues from a project whose resolution field has been hidden. A hidden field is assumed not to exist.
There are some extra interfaces that the
CustomFieldSearcherClauseHandler may also implement to provide optional functionality to the searching subsystem:
- ValueGeneratingClauseHandler: Gives the
CustomFieldSearcherthe ability to suggest some values during JQL entry auto-complete. This is really only useful for custom fields whose values come from an allowable finite set.
- CustomFieldClauseSanitiserHandler: Gives the
CustomFieldSearcherthe ability pre-process the query and remove sensitive information from the query before it is displayed to the passed user.
- CustomFieldClauseContextHandler: Gives the
CustomFieldSearcherthe ability to customise JIRA's query context calculation. This interface is best left alone, unexplained and unimplemented.