Jira stays in the Upgrade Mode due to the failed Upgrade Task [host,buildNumber=814001]
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Jira stays in the Upgrade Mode due to the failed Upgrade Task [host,buildNumber=814001]
Environment
Jira 8.20 and newer.
Diagnosis
In atlassian-jira.log a message like the below is displayed:
2022-01-04 16:40:49,064+0100 Caesium-1-4 ERROR ServiceRunner [c.a.upgrade.core.DefaultUpgradeTaskFactoryProcessor] Upgrade task [host,buildNumber=814001] failed
java.lang.RuntimeException: Migration failed. Could not create directory for encryption keys.
at com.atlassian.jira.upgrade.tasks.UpgradeTask_Build814001.createKeysDirectory(UpgradeTask_Build814001.java:134)
at com.atlassian.jira.upgrade.tasks.UpgradeTask_Build814001.encryptPasswordsAndSaveConfiguration(UpgradeTask_Build814001.java:117)
at com.atlassian.jira.upgrade.tasks.UpgradeTask_Build814001.runUpgrade(UpgradeTask_Build814001.java:94)
at com.atlassian.upgrade.core.DefaultUpgradeTaskFactoryProcessor.runOneUpgradeTask(DefaultUpgradeTaskFactoryProcessor.java:109)
at com.atlassian.upgrade.core.DefaultUpgradeTaskFactoryProcessor.lambda$performUpgradesUnsafe$13(DefaultUpgradeTaskFactoryProcessor.java:80)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:176)
at java.base/java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:400)
at java.base/java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:503)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:543)
at com.atlassian.upgrade.core.DefaultUpgradeTaskFactoryProcessor.performUpgradesUnsafe(DefaultUpgradeTaskFactoryProcessor.java:81)
at com.atlassian.upgrade.core.DefaultUpgradeTaskFactoryProcessor.performUpgrades(DefaultUpgradeTaskFactoryProcessor.java:46)
at com.atlassian.upgrade.core.DefaultUpgradeTaskManager.upgradeHostApp(DefaultUpgradeTaskManager.java:41)
at com.atlassian.jira.upgrade.LicenseCheckingUpgradeService.executeUpgrades(LicenseCheckingUpgradeService.java:134)
at com.atlassian.jira.upgrade.LicenseCheckingUpgradeService.runUpgrades(LicenseCheckingUpgradeService.java:97)
at com.atlassian.jira.upgrade.ClusterLockingUpgradeService.runUpgrades(ClusterLockingUpgradeService.java:35)
at com.atlassian.jira.upgrade.LoggingUpgradeService.lambda$runUpgradesWithLogging$0(LoggingUpgradeService.java:28)
at com.atlassian.jira.upgrade.LoggingUpgradeService.runWithTaskLogging(LoggingUpgradeService.java:43)
at com.atlassian.jira.upgrade.LoggingUpgradeService.runUpgradesWithLogging(LoggingUpgradeService.java:28)
at com.atlassian.jira.upgrade.IndexingUpgradeService.runUpgrades(IndexingUpgradeService.java:19)
at com.atlassian.jira.upgrade.UpgradeScheduler.runHostUpgrades(UpgradeScheduler.java:100)
at com.atlassian.jira.upgrade.UpgradeScheduler.runUpgrades(UpgradeScheduler.java:80)
at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:134)
at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:106)
at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:90)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:435)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:430)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:454)
at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:382)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:66)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:60)
at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:35)
at java.base/java.lang.Thread.run(Thread.java:829)Cause
This task is related to a new feature that was introduced to Jira that allows us to encrypt the password from the external directories at the database. In most cases, Jira OS user does not have the proper permissions in the Jira_Home, Jira_Install, and Shared folders.
Solution
First of all, ensure that the Jira OS user has all the necessary (read/write/execute) permissions. Please also see: How to fix directory permissions in Linux for Jira Server. If you still have this problem even the related user has the proper permissions, then you can skip running that task by following the steps below:
- Stop both nodes
- Take a Jira database backup
- Connect to Jira database and run the statements below:
INSERT INTO propertyentry (id, entity_name, entity_id, property_key, propertytype) values ((SELECT id+1 from propertyentry order by id desc limit 1), 'jira.properties', 1, 'crowd.encryption.encryptor.default', 5);
INSERT INTO propertystring (id, propertyvalue) values ((SELECT id+1 from propertyentry where property_key='crowd.encryption.encryptor.default'), 'DISABLED');`Once you are done with the upgrade, you can enable this disabled feature by following the steps mentioned in Configuring advanced settings and setting crowd.encryption.encryptor.default to AES_CBC_PKCS5Padding. You don't need to restart Jira after changing this configuration.