Bamboo fails to start: Unable to clean the Felix cache directory (Task 51404)
Summary
As a Bamboo instance is initiated there are several validation tasks are run before bootstrap upgrade tasks. If any of them fails, it will report an error and the application will not run.
Environment
All Bamboo versions.
Diagnosis
This issue is related to the failure of Task 51404 present at the Bamboo start-up. This task makes sure that the bundle cache directory is empty under the path <bamboo-home>/caches/plugins/felix/felix-cache
. If Bamboo cannot clear the cache, the bootstrap fails and throws the errors below.
INFO [localhost-startStop-1] [BootstrapUpgradeManagerImpl] ---------------------------------------------------------------------------------------
INFO [localhost-startStop-1] [BootstrapUpgradeManagerImpl] 51404 : Make sure OSGi bundle cache directory is empty, clean it if needed. (bootstrap)
INFO [localhost-startStop-1] [BootstrapUpgradeManagerImpl] ---------------------------------------------------------------------------------------
INFO [localhost-startStop-1] [BootstrapUpgradeManagerImpl] Task https://confluence.atlassian.com/x/ehivEg failed.
Issue 1
The UI shows the following error message:
Your Bamboo instance could not start because health check failed. Contact Atlassian Support at https://support.atlassian.com
When looking at the atlassian-bamboo.log the following error message appears: Task 51404 failed - java.io.IOException: Failed to list contents of /bamboo/caches/plugins/felix/felix-cache
ERROR [localhost-startStop-1] [BootstrapUpgradeManagerImpl] Task 51404 failed
java.io.IOException: Failed to list contents of /bamboo/caches/plugins/felix/felix-cache
at org.apache.commons.io.FileUtils.verifiedListFiles(FileUtils.java:1751)
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1716)
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1617)
....
2022-05-23 10:32:13,447 FATAL [localhost-startStop-1] [DefaultBootstrapManager] Validation tests failed: [Task 51404 failed with exception: Failed to list contents of /bamboo/caches/plugins/felix/felix-cache]
2022-05-23 10:32:13,519 FATAL [localhost-startStop-1] [DefaultBootstrapManager] Validation tests failed: Your Bamboo instance could not start because health check failed. Contact Atlassian Support at https://support.atlassian.com
Issue 2
Another error message seen in the logs is "Unable to clean the cache directory: D:\bamboo\caches\plugins\felix"
:
ERROR [WrapperSimpleAppMain] [ContextLoader] Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'capabilityHelper' defined in class path resource [applicationContext.xml]: Cannot resolve reference to bean 'pluginAccessor' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'pluginManager' defined in class path resource [applicationContextPlugin.xml]: Cannot resolve reference to bean 'bundledPluginLoader' while setting constructor argument with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'bundledPluginLoader' defined in class path resource [applicationContextPluginCommon.xml]: Cannot resolve reference to bean 'osgiPluginFactory' while setting constructor argument with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'osgiPluginFactory' defined in class path resource [applicationContextPluginCommon.xml]: Cannot resolve reference to bean 'osgiPersistentCache' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'osgiPersistentCache' defined in class path resource [applicationContextPluginCommon.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.plugin.osgi.container.impl.DefaultOsgiPersistentCache]: Constructor threw exception; nested exception is com.atlassian.plugin.osgi.container.OsgiContainerException: Unable to clean the cache directory: D:\bamboo\caches\plugins\felix
...
Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.plugin.osgi.container.impl.DefaultOsgiPersistentCache]: Constructor threw exception; nested exception is com.atlassian.plugin.osgi.container.OsgiContainerException: Unable to clean the cache directory: D:\bamboo\caches\plugins\felix
Caused by: com.atlassian.plugin.osgi.container.OsgiContainerException: Unable to clean the cache directory: D:\bamboo\caches\plugins\felix
at com.atlassian.plugin.osgi.container.impl.DefaultOsgiPersistentCache.validate(DefaultOsgiPersistentCache.java:91)
at com.atlassian.plugin.osgi.container.impl.DefaultOsgiPersistentCache.<init>(DefaultOsgiPersistentCache.java:37)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
...
Caused by: java.io.IOException: Unable to delete file: D:\bamboo\caches\plugins\felix\felix-cache\bundle9\version0.0\bundle.jar
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1393)
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1047)
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:978)
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1384)
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1047)
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:978)
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1384)
...
Cause
Bamboo cannot delete a file from the <BAMBOO_HOME>/caches
directory.
Resolution
The contents of the directory <BAMBOO_HOME>/caches
have to be manually removed before starting Bamboo. To do that:
- Shut down the Bamboo instance.
- Manually delete contents inside the <
BAMBOO_HOME>/caches
directory. - Start Bamboo.
If Bamboo was previously started as a different user, such as root
, many files in the BAMBOO_HOME
directory will be owned by root. If you noticed this issue after that happened, please be sure to recursively change the ownership of that directory and its files back to your Bamboo user.