Jira fails to startup and gets locked due to slow CPU
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Problem
JIRA might fail to start-up and show JIRA Locked error. This is caused by slow initialization of plugin framework due to not enough CPU resources (slow or throttled CPU)
The following appears in the atlassian-jira.log
2018-05-28 10:32:25,811 JIRA-Bootstrap INFO [c.a.plugin.manager.DefaultPluginManager] Plugin system earlyStartup begun
2018-05-28 10:34:24,552 ThreadPoolAsyncTaskExecutor::Thread 9 WARN [o.s.b.factory.support.DefaultListableBeanFactory] Overriding user-defined bean definition for bean 'eventPublisher' with a framework-generated bean definition: replacing [Generic bean: class [com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean]; scope=; abstract=false; lazyInit=true; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in URL [bundle://51.0:0/META-INF/spring/atlassian-plugins-host-components.xml]] with [Root bean: class [org.eclipse.gemini.blueprint.service.importer.support.OsgiServiceProxyFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null]
...
2018-05-28 10:35:01,135 JIRA-Bootstrap INFO [c.a.plugin.util.WaitUntil] Plugins that have yet to be enabled: (117):
2018-05-28 10:35:06,702 Spring DM Context Creation Timer WARN [o.e.g.b.e.i.dependencies.startup.DependencyWaiterApplicationContextExecutor] Timeout occurred before finding service dependencies for [NonValidatingOsgiBundleXmlApplicationContext(bundle=com.atlassian.administration.atlassian-admin-quicksearch-jira, config=osgibundle:/META-INF/spring/*.xml)]
2018-05-28 10:35:06,715 Spring DM Context Creation Timer ERROR [c.a.p.osgi.factory.OsgiPlugin] Plugin 'com.atlassian.administration.atlassian-admin-quicksearch-jira' never resolved service '&sal-request-factory-import' with filter '(objectClass=com.atlassian.sal.api.net.RequestFactory)'
....
The following plugins are required by JIRA, but have not been started:
JIRA Projects Plugin (com.atlassian.jira.jira-projects-plugin), Atlassian Navigation Links Plugin (com.atlassian.plugins.atlassian-nav-links-plugin), Gadget Directory Plugin (com.atlassian.gadgets.directory), Atlassian JIRA - Plugins - Gadgets Plugin (com.atlassian.jira.gadgets), Atlassian JIRA - Plugins - Global Issue Navigator (com.atlassian.jira.jira-issue-nav-plugin), Atlassian JIRA - Plugins - Quick Edit Plugin (com.atlassian.jira.jira-quick-edit-plugin), Atlassian JIRA - Plugins - REST Plugin (com.atlassian.jira.rest), Gadget Dashboard Plugin (com.atlassian.gadgets.dashboard)
2018-05-28 10:37:26,322 JIRA-Bootstrap WARN [c.a.jira.startup.JiraStartupLogger]
___ FAILED PLUGIN REPORT _____________________
49 plugins failed to load during JIRA startup.Diagnosis
Environment
- Slow or busy CPU
Diagnostic Steps
- Check time between plug-in initialization stages (from the snippet above):
2018-05-28 10:32:25,811 JIRA-Bootstrap INFO [c.a.plugin.manager.DefaultPluginManager] Plugin system earlyStartup begun
2018-05-28 10:34:24,552 ThreadPoolAsyncTaskExecutor::Thread 9 WARN [o.s.b.factory.support.DefaultListableBeanFactory]
2018-05-28 10:35:01,135 JIRA-Bootstrap INFO [c.a.plugin.util.WaitUntil] Plugins that have yet to be enabled: (11- Note the timestamps: 10:32:25, 10:34:24, 10:35:01 - difference is quite large.
- Happy flow - it took 14 seconds for the first message and 20 seconds to have more plugins enabled:
2018-05-28 11:02:20,315 JIRA-Bootstrap INFO [c.a.plugin.manager.DefaultPluginManager] Plugin system earlyStartup begun
2018-05-28 11:02:34,709 ThreadPoolAsyncTaskExecutor::Thread 4 WARN [o.s.b.factory.support.DefaultListableBeanFactory] Overriding user-defined
2018-05-28 11:02:40,500 JIRA-Bootstrap INFO [c.a.plugin.util.WaitUntil] Plugins that have yet to be enabled: (98):Cause
JIRA Plugin manager uses different timeouts to initialize the OSGi and other frameworks. If CPU is slow those timeouts expire and that leads to not able to load important libraries, eg: com.atlassian.sal.api.net.RequestFactory. That leads to cascading failure effect and massive errors with plugin loading.
Resolution
Since the problem caused by CPU contention, this should be the main focus. A couple of suggestions to do/check:
- Check that there are no other CPU active processes
- Check how CPU your system has, add more CPU or remove CPU throttle if required
After that got resolved, restarting the JIRA will fix the problem.
Additionally, it's possible to increase the startup time for plugins: Plugins require more time to startup