Confluence Data Center will not start up because the build number in the Home Directory doesn't match the build number in the Database, after upgrade
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
When attempting to upgrade to Confluence version 5.1.2 or higher, the upgrade fails and the following errors appear in the atlassian-confluence.log :
20XX-01-01 00:00:00,00 ERROR [main] [atlassian.confluence.setup.BootstrapApplicationStartupListener] checkConfigurationOnStartup Confluence will not start up because the build number in the Home Directory [<build number>] doesn't match the build number in the Database [<build number>].20XX-01-01 00:00:00,00 ERROR [Catalina-utility-1] [atlassian.confluence.setup.BootstrapApplicationStartupListener] checkConfigurationOnStartup Confluence cluster node will not start up because the finalized build number in the database [<build number>] doesn't match either the application build number [<build number>] or the home directory build number [<build number>].The versions represented by the build numbers can be found on this page: Confluence Build Information.
Cause
This can be caused by one of the following situations:
- If you have just performed a site restore and you are running Confluence 5.1.4 or earlier, you may be encountering this bug: CONF-29758 - Getting issue details... STATUS If your situation matches what is discussed in the description, please follow the workaround provided in that bug report.
- At some point in the past an upgrade failed.
- The database has been restored to an earlier version, but the home directory hasn't, or vice versa.
In all cases, it means that not all upgrade tasks have run on both the database and the home directory.
The information about the version of Confluence is stored in the database in the CONFVERSION table and in the home directory in <confluence_home_directory>/confluence.cfg.xml
The CONFVERSION table, which stores information about the versions of Confluence your instance has been upgraded to, is actually a record of the upgrade tasks that have completed successfully. When an upgrade task completes, Confluence puts an entry in the CONFVERSION table with the build number of that upgrade task.
This is why the build numbers in the CONFVERSION table sometimes don't match the actual build numbers of releases; each upgrade task has a build number, and then each release has a build number that is higher than all the upgrade tasks that are required to upgrade to that version.
Resolution
You can find the build numbers for:
The database by running:
select * from CONFVERSION;The home directory by looking in the
<confluence_home_directory>/confluence.cfg.xmlfile:<confluence-configuration> ... <buildNumber>4215</buildNumber> <properties> ... ... <property name="finalizedBuildNumber">4215</property> ... ... </properties> </confluence-configuration>
Resolution 1
If possible, both the database and confluence.cfg.xml file within the home directory should be restored to a point before the failed upgrade attempt. Once this has been done, ensure that the buildnumber field within the CONFVERSION table and the buildNumber tag (along with the finalizedBuildNumber tag for newer versions of Confluence) in the confluence.cfg.xml file within the home directory are identical. And if a Data Center cluster is enabled, also ensure that the buildNumber tag of the confluence.cfg.xml file that's located in the shared home folder matches.
Resolution 2
If restoring to that point is not possible, manually set the build numbers back to the lower value of the two.
For example, if the buildNumber tag of the confluence.cfg.xml file that's located in the local home folder shows 4215, and the latest buildnumber field within the CONFVERSION table database shows 3289, then the buildNumber tag of the confluence.cfg.xml file should be set to 3289 and then the upgrade should be performed again. This will trigger all applicable upgrade tasks between the Confluence version that the database is using and the version of the installation files that are being upgraded to, resulting in possible updates to the database schema and updates to the build number tags within the confluence.cfg.xml file.
If a Data Center cluster is enabled, then the buildNumber tag of the confluence.cfg.xml file that's located in the shared home folder will also need to be reset prior to reattempting the upgrade. Also, for newer versions of Confluence, please also reset the finalizedBuildNumber tag of the confluence.cfg.xml file that's located in the local home folder.
To change the value in the Database, delete any entries that have a build number higher than the version in your home directory:
Under no circumstances should one remove all rows/version information from the CONFVERSION table
delete from CONFVERSION where BUILDNUMBER > YOUR_HOME_DIRECTORY_BUILD_NUMBERResolution 3
First, let's see what the last record looks like in the CONFVERSION table - the build number here should be the same as that build number referenced for the DB in the message:
select * from CONFVERSION;If you note that the confluence.cfg.xml file contains the upgraded database build number, but the CONFVERSION table shows the row similar to the example below:
| CONFVERSIONID | BUILDNUMBER | INSTALLDATE | VERSIONTAG | CREATIONDATE | LASTMODDATE |
|---|---|---|---|---|---|
| 4560123456 | 6441 | 2017-07-04 06:01:42.54 | Locked for version upgrade to 7201 |
This suggests the schema upgrades have not completed. This issue can be corrected manually by reverting back the build numbers and using the -Datlassian.forceSchemaUpdate system property.
Update tags within the confluence.cfg.xml file(s)
Begin by updating build number in the buildNumber tag of the confluence.cfg.xml file that's located in the local home folder (and shared home folder, if a Data Center cluster is configured). The build number should match the database build number (ie 6441 in the above scenario). Also, for newer versions of Confluence, please also update the finalizedBuildNumber tag of the confluence.cfg.xml file that's located in the local home folder.
Update the versiontag field in the CONFVERSION table
Next, the versiontag field for the most recent build number should be set to null if it is not:
Database Updates:
As this involved modification to the database, you are strongly advised to create a backup of your database prior to modification
Also: Please remember to COMMIT the changes (Oracle requires the changes to be committed before they are accepted)
UPDATE CONFVERSION
SET VERSIONTAG = NULL
WHERE CONFVERSIONID = 4560123456;Regarding the "lock_for_upgrade_to_" message:
- It is inserted into the CONFVERSION table only for clustered Data Center instances and acts as a database row lock in order to ensure that only one node within a cluster is initiating schema changes to the database.
- It is inserted even though there might not be a change to a build number.
- It's OK for it to remain in the table after an upgrade has successfully completed.
- It needs to be reset to NULL or some other value only in the event of reattempting a failed upgrade.
- Other records referencing intermediate build numbers may be inserted into the table during an upgrade. Their entry is triggered by various upgrade tasks within the overall upgrade process.
Add the forceSchemaUpdate Java option
Finally, to force a schema upgrade apply the following system property to the instance (see Configuring System Properties for OS-specific steps): -Datlassian.forceSchemaUpdate=true
Once these changes have been made stop all nodes before allowing a single node to start fully. Then confirm the build numbers match across the database and the confluence.cfg.xml, and if that is the case, the -Datlassian.forceSchemaUpdate=true can be removed.