Various operations fail in JIRA on Oracle Database
Symptoms
Various JIRA operations fail with various errors (mostly 500) and and exceptions like below appear in the atlassian-jira.log
:
2020-06-02 01:22:05,428+0000 http-nio-8080-exec-2 WARN xxx 82x17369x1 1qon4dl x.x.x.x /secure/admin/ViewLogging.jspa [webresource] exception thrown in `addToUrl` during condition evaluation
com.atlassian.activeobjects.internal.ActiveObjectsInitException: bundle [com.atlassian.whisper.atlassian-whisper-plugin]
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:95)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:86)
at com.atlassian.sal.core.executor.ThreadLocalDelegateCallable.call(ThreadLocalDelegateCallable.java:38)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.NullPointerException
at org.ofbiz.core.entity.jdbc.dbtype.DatabaseTypeFactory.logDatabaseNotMatchedError(DatabaseTypeFactory.java:85)
at org.ofbiz.core.entity.jdbc.dbtype.DatabaseTypeFactory.getTypeForConnection(DatabaseTypeFactory.java:79)
at com.atlassian.activeobjects.jira.OfBizDatabaseTypeExtractor.getDatabaseType(OfBizDatabaseTypeExtractor.java:13)
at com.atlassian.activeobjects.jira.JiraDataSourceProvider.lambda$getDatabaseType$0(JiraDataSourceProvider.java:62)
at com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at com.atlassian.activeobjects.jira.JiraTenantAwareDataSourceProvider.getDatabaseType(JiraTenantAwareDataSourceProvider.java:46)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy1158.getDatabaseType(Unknown Source)
at com.atlassian.activeobjects.internal.DataSourceProviderActiveObjectsFactory.getDatabaseType(DataSourceProviderActiveObjectsFactory.java:97)
at com.atlassian.activeobjects.internal.DataSourceProviderActiveObjectsFactory.upgrade(DataSourceProviderActiveObjectsFactory.java:76)
at com.atlassian.activeobjects.internal.AbstractActiveObjectsFactory.create(AbstractActiveObjectsFactory.java:65)
at com.atlassian.activeobjects.internal.DelegatingActiveObjectsFactory.create(DelegatingActiveObjectsFactory.java:32)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:91)
... 6 more
While the NullPointer exception usually indicates there's a NULL value somewhere in the database, in this specific situation it's a red herring and we need to check what had happened (or had been happening prior to the exception). If we see the following exception in the logs prior to the NullPointer, then JIRA is affected by the issue:
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.sql.SQLSyntaxErrorException: ORA-00923: FROM keyword not found where expected
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:509)
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:461)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1104)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:550)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:655)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:270)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:91)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:807)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:983)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1168)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3666)
at oracle.jdbc.driver.T4CPreparedStatement.executeInternal(T4CPreparedStatement.java:1426)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3713)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1167)
at org.apache.commons.dbcp2.PoolableConnection.validate(PoolableConnection.java:301)
at org.apache.commons.dbcp2.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:357)
at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2304)
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2287)
... 60 more
Caused by: Error : 923, Position : 8, Sql = select 1, OriginalSql = select 1, Error Msg = ORA-00923: FROM keyword not found where expected
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:513)
... 78 more
Cause
The cause here is documented in this article , most likely the validation query is set to 'SELECT 1' which is incorrect for Oracle. 'select 1 from dual' should be used instead.
Resolution
Ensure the database has been configured as per the sample XML in Connecting JIRA to Oracle.
Specific attention must be paid to the validation query parameters. If these are not configured correctly it will cause this problem.