Skip to end of metadata
Go to start of metadata

Overview

In Agile 6.4 and above, the ranking system has been changed to support cluster-safe rank operations on JIRA Data Center. The new rank system (often called 'LexoRank') is stored in a lexicographical order of strings to determine the ranking of issues.

In the course of this article, some specific terms will be used:

  • Re-balance job
    This is a plugin job to maintain the integrity of the ranking table. The JIRA Agile plugin should run this job automatically. This is a background job, which means it should not interfere with the user operation.
  • Rank bucket
    The new ranking system utilizes 3 'buckets' (0, 1, 2) to move issues around during the re-balance job. The bucket is indicated by the number prefix before the pipeline character ('|') in the Rank field value:
    "0|...." indicates ranking is stored in bucket 0, "1|..." indicates bucket 1, and so on.

Troubleshooting Steps

  1. Check the database collation and encoding setting, as per Configuring Database Character Encoding.
  2. Ensure you are running on the latest Agile version.
  3. Run Rank Integrity Checker by accessing https://<jira-base-url>/rest/greenhopper/1.0/lexorank/integrity (perform a GET, this is done if you access the URL in the browser).

     Click here to expand for sample integrity results...
    {
      "reports": [
        {
          "rankFieldId": 15201,
          "rankFieldName": "Global Rank",
          "results": [
            {
              "name": "Marker rows present in table for rank field",
              "description": "Checks if the rank table has been properly initialized for the rank field. A minimum and maximum marker row are expected to be present in the table for the rank field.",
              "passed": true,
              "fatal": true
            },
            {
              "name": "Marker rows correctness check",
              "description": "Checks whether the marker rows for a rank field have the expected rank value.",
              "passed": true,
              "fatal": true
            },
            {
              "name": "Marker rows in valid bucket check.",
              "description": "Checks if the marker rows of a rank field are in valid bucket(s).",
              "passed": true,
              "fatal": true
            },
            {
              "name": "Rank out of bounds check",
              "description": "Checks if there is a rank value for the rank field that is out of bounds.",
              "passed": true,
              "fatal": true
            },
            {
              "name": "Duplicate ranks check",
              "description": "Checks if there are any duplicate rank values for a rank field.",
              "passed": false,
              "failureReason": "Detected duplicate ranks for rank field 15201 : {0|11t9b4:=2, 0|11sqnc:=2, 0|11suow:=2, 0|11tm0w:=2, 0|11thk8:=2, 0|11spog:=2, 0|11tes0:=2, 0|11sx00:=2, 0|11t2qg:=2, 0|11sqx4:=2, 0|11tde8:=2, 0|11t9zc:=2, 0|11sxcw:=2, 0|11ssi0:=2, 0|11teqo:=2, 0|11tlm0:=2, 0|11t140:=2, 0|11tbp4:=2, 0|11tg5s:=2, 0|11srhs:=2, 0|11t03s:=2, 0|11t4bc:=2, 0|11taow:=2, 0|11td48:=2}",
              "fatal": false
            },
            {
              "name": "Issue ranks different from marker ranks check",
              "description": "Checks if there are any issue ranks that have the same rank as the maximum or minimum rank.",
              "passed": true,
              "fatal": true
            },
            {
              "name": "Issue rows in valid bucket check.",
              "description": "Checks if the issue rows of a rank field are in a valid bucket. Issue rows should be in one of the buckets the marker rows are in.",
              "passed": true,
              "fatal": false
            },
            {
              "name": "Balance status check",
              "description": "Checks if there is a correct balance entry is present or absent for the rank field.",
              "passed": true,
              "fatal": false
            }
          ]
        }
  4. Depending on which test is failing, check with the KB article section down below.

  5. If you are in doubt, please follow sending information to support.

Sending Information to Support

Please raise a case at https://support.atlassian.com and provide the following information:

  1. Generate JIRA support zip.
  2. The result of rank integrity checks.
  3. An XML backup of your database (Administration > System > Backup System). You can anonymise if you wish by going as per our Anonymising JIRA Data page.
  4. If attaching a backup is not possible, please provide us with the result of the following SQL (preferably in a CSV file format).

    SELECT * FROM "AO_60DB71_LEXORANK" ORDER BY "RANK";

    (info) Depending upon the DBMS used, the tablename may be different. For example AO_60DB71_LEXORANK instead of "AO_60DB71_LEXORANK".

Enable Additional Logging for LexoRank

By default, the logging of ranking system is disabled. To get detailed logging of what the system is doing add a new logger to JIRA's Logging and Profiling page

  1. Logging for balancing
    level: DEBUG
    package: com.atlassian.greenhopper.service.lexorank.balance
  2. Logging for ranking
    level: DEBUG
    package: com.atlassian.greenhopper.service.lexorank
  3. Logging for statistics: every minute logs the activity and performance of the ranking system
    level: DEBUG
    package: com.atlassian.greenhopper.service.lexorank.LexoRankStatisticsAgent

FAQ

How to check if a re-balance is scheduled?

When a re-balance is scheduled, the AO_60DB71_LEXORANKBALANCER table in the database should contain an entry for each Rank field.

The REBALANCE_TIME column will contain the time when the operation is scheduled to start in Epoch time (aka Unix time).

How to check if a re-balance is running?

Accessing https://<jira-base-url>/rest/greenhopper/1.0/lexorank/balance through the browser will return information on the status of the balancer. This will show you whether or not it's running, and also a % status and the distribution of the buckets.

 Click here to expand for sample integrity results...
{
  "lexoRankBalancingServiceStatus": {
    "balancingDisabled": false,
    "balanceHandlerRunning": false
  },
  "lexoRankBalancerStatus": {
    "balancerLocked": false,
    "perFieldStatus": [
      {
        "fieldName": "Rank",
        "fieldId": 10105,
        "numRankedIssues": 1500,
        "percentComplete": 100,
        "distribution": [
          1500,
          0,
          0
        ]
      }
    ]
  }
}

A re-balance will be running when the following conditions are met:

  1. There is an entry in the AO_60DB71_LEXORANKBALANCER table in the database, scheduled for a time which has already passed.
  2. The query below should return two rows. They must be on different buckets during a rebalance, on one the following bucket combinations: 0 and 1; 1 and 2; 2 and 0
    (warning) Once a rebalance operation is completed, this query will return two ranks on the same bucket (e.g. "1|000000:" and "1|zzzzzz:").
SELECT "RANK" FROM "AO_60DB71_LEXORANK" WHERE "TYPE" <> 1 ORDER BY "RANK";

When re-balance job is triggered or required?

  1. When there are new issues created with no ranking assigned.
  2. When the rank character length exceeds 50 characters.
  3. When the the ranking value splits in 2 different buckets.

What happens if we stop JIRA while a re-balance is happening?

Whenever the JIRA Agile plugin is stopped, JIRA is stopped or a foreground re-index is triggered, the LexoRank scheduler job will stop any balance job that is running. Then the actual plugin scheduler job is stopped by the plugin system. Once the plugin is re-enabled, the balance job will start again and pick up from where it left. It will check the AO_60DB71_LEXORANKBALANCER table again and see that there are still entries for balance job to be run and start going through them 1 by 1. Once a balance job is completed, the entry in the table will be removed.

Can JIRA Agile be updated while a re-balance is happening?

Yes. However, we highly recommend taking a backup from the JIRA instance prior to updating the plugin. Additionally, updating JIRA Agile might not necessarily fix existing inconsistencies in the LexoRank system in case it is already corrupted. In such cases, please report a support ticket providing the information requested on the Sending Information to Support section.

 

Lexorank KB Articles

2 Comments

  1. When a rebalance is running the REBALANCE_TIME column seems to contain a number so far in the future it is treated as being in the past by Agile 

    1. Actually, REBALANCE_TIME is in ms so remove the last 3 digits before using http://www.onlineconversion.com/unix_time.htm to get the clock time