Jira Data Center node locking on startup

Still need help?

The Atlassian Community is here for you.

Ask the community


Platform Notice: Data Center Only - This article only applies to Atlassian products on the data center platform.

Problem

When starting up a Jira node in cluster, and it is not coming up successfully, there is a FATAL error in the log: 

2020-07-28 16:34:37,899 Jira-Bootstrap FATAL      [c.a.jira.startup.JiraStartupLogger] Startup check failed. Jira will be locked.
  • this means that the startup process failed to complete all the startup checks

  • there are checks on db driver, database access and cluster access

  • shared home is an important part of cluster access

The following appears in the catalina.out log:

28-Jul-2020 16:34:37.589 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
28-Jul-2020 16:34:37.596 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
28-Jul-2020 16:34:37.611 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 4293 ms
2020-07-28 16:34:37,723 Jira-Bootstrap INFO      [c.a.j.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /appl/jira/home/dbconfig.xml
2020-07-28 16:34:37,899 Jira-Bootstrap INFO      [c.a.jira.startup.JiraStartupLogger] Running Jira startup checks.
2020-07-28 16:34:37,899 Jira-Bootstrap FATAL      [c.a.jira.startup.JiraStartupLogger] Startup check failed. Jira will be locked.
2020-07-28 16:34:37,960 Jira-Bootstrap INFO      [c.a.jira.startup.LauncherContextListener] Memory Usage:

Diagnosis

Environment

  • Database driver is present and there are no problems connecting to the database 

Diagnostic Steps

  • A related symptom is that the application logs are not being written to

  • the Jira process user already has full permissions to local node installation and Jira home directory


Happy flow

In case of all checks are successful, there will be logging similar to this:

2020-01-13 17:08:12,779+0100 localhost-startStop-1 INFO      [c.a.jira.startup.JiraHomeStartupCheck] The jira.home directory '/app/jira/application-data/jira_8.5' is validated and locked for exclusive use by this instance.


Validation code 

Checks that are executed as part of validateJiraHome

 // Validate the jira.home, and create the directory if required
 // Note that we only save jiraHomeDirectory if everything is valid.
    private Either<JiraHomeStartupCheckFailure, File> validateJiraHome(final String jiraHome) {
        final File proposedJiraHome = new File(jiraHome);
        return checkProposedHomeAbsolute(proposedJiraHome, jiraHome)
                .flatMap(this::checkWebappPathCollision)
                .flatMap(this::checkJiraHomeCreationIsPossible)
                .flatMap(this::createJiraHomeAndSharedHome)
                .flatMap(this::lockJiraHome);
    }


Cause

There is a problem with the shared home access. Shared home in Jira Data Center needs to be accessed read/write by the Jira process. Additionally, if the shared home is too slow (latency) or intermittently unavailable, the same problem may be observed.

Subdirectories that checked/created as part of createSharedHomeDirectories

private Either<JiraHomeStartupCheckFailure, File> createSharedHomeDirectories(final File sharedJiraHome, final File proposedJiraHome) {
        final Set<String> subdirs = CollectionBuilder.<String>newBuilder()
                .add(AttachmentPathManager.ATTACHMENTS_DIR)
                .add(PluginPath.PLUGINS_DIRECTORY)
                .add(PluginPath.BUNDLED_SUBDIR)
                .add(PluginPath.OSGI_SUBDIR)
                .add(FileService.MAIL_DIR)
                .addAll(JiraHome.sharedsubdirs)
                .asMutableSortedSet();


Resolution

To fix the problem on startup:

  • Verify that the database driver and access are valid on the node

  • Ensure that all Jira folders are owned and writable by the Jira process user

  • Validate and fix any problem with shared home read/write access 


DescriptionJira DC startup problem
ProductJira Data Center
Last modified on Nov 23, 2020

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.