Oracle database errors after upgrading Bitbucket Data Center or Server
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Audit log entries are not displayed in the Bitbucket's web interface after upgrading from version 7.4 to 7.9.
Errors in Bitbucket's application logs indicate that Oracle is not able to extend the size for its tablespace, as it has reached the maximum defined size.
Please note that these errors may also occur independently from upgrading Bitbucket.
Environment
Bitbucket Data Center version 7.9.1 (other versions may also run into this issue)
Oracle 12c database
Diagnostic
In Bitbucket's application log (atlassian-bitbucket.log
) the following error was found:
2021-04-17 05:51:21,134 ERROR [https-jsse-nio-8443-exec-5] username @7I3MC2x351x1084x1 n9eru0 10.80.201.103 "GET /plugins/servlet/audit/undefined/rest/shortcuts/latest/shortcuts/738dc90/bundled HTTP/1.1" c.a.a.f.d.AuditEventsViewDataProvider Failed to get action filters
...
Caused by: com.atlassian.activeobjects.internal.ActiveObjectsSqlException: There was a SQL exception thrown by the Active Objects library:
Database:
- name:Oracle
- version:Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
- minor version:1
- major version:12
Driver:
- name:Oracle JDBC driver
- version:12.2.0.1.0
java.sql.SQLException: ORA-01652: unable to extend temp segment by 128 in tablespace STASHTBS
Because of these errors, creation of an index for the "AO_C77861_AUDIT_ENTITY" table, which contains the audit log entries, failed.
Bitbucket's application logs include error creating index "index_ao_c77861_aud265617021" for "AO_C77861_AUDIT_ENTITY" table:
2021-04-17 05:32:19,154 ERROR [active-objects-init-compatibility-tenant-0] net.java.ao.sql Exception executing SQL update <CREATE INDEX "index_ao_c77861_aud265617021" ON "AO_C77861_AUDIT_ENTITY"("ACTION")>
java.sql.SQLException: ORA-01652: unable to extend temp segment by 128 in tablespace STASHTBS
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:494)
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:446)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1054)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:623)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:213)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:37)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:896)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1119)
at oracle.jdbc.driver.OracleStatement.executeUpdateInternal(OracleStatement.java:1661)
at oracle.jdbc.driver.OracleStatement.executeLargeUpdate(OracleStatement.java:1626)
at oracle.jdbc.driver.OracleStatement.executeUpdate(OracleStatement.java:1613)
at oracle.jdbc.driver.OracleStatementWrapper.executeUpdate(OracleStatementWrapper.java:282)
at com.zaxxer.hikari.pool.ProxyStatement.executeUpdate(ProxyStatement.java:120)
at com.zaxxer.hikari.pool.HikariProxyStatement.executeUpdate(HikariProxyStatement.java)
at net.java.ao.DatabaseProvider.executeUpdate(DatabaseProvider.java:2354)
at net.java.ao.DatabaseProvider.executeUpdateForAction(DatabaseProvider.java:2410)
at net.java.ao.DatabaseProvider.executeUpdatesForActions(DatabaseProvider.java:2382)
at net.java.ao.schema.SchemaGenerator.migrate(SchemaGenerator.java:91)
at net.java.ao.EntityManager.migrate(EntityManager.java:131)
at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.migrate(EntityManagedActiveObjects.java:52)
at com.atlassian.activeobjects.internal.AbstractActiveObjectsFactory.lambda$create$0(AbstractActiveObjectsFactory.java:73)
at com.atlassian.sal.core.transaction.HostContextTransactionTemplate$1.doInTransaction(HostContextTransactionTemplate.java:21)
at com.atlassian.stash.internal.sal.spi.HostContextAccessorImpl.doInTransaction(HostContextAccessorImpl.java:54)
at com.atlassian.sal.core.transaction.HostContextTransactionTemplate.execute(HostContextTransactionTemplate.java:18)
at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
at com.atlassian.activeobjects.internal.AbstractActiveObjectsFactory.create(AbstractActiveObjectsFactory.java:70)
at com.atlassian.activeobjects.internal.DelegatingActiveObjectsFactory.create(DelegatingActiveObjectsFactory.java:32)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:91)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:86)
at com.atlassian.sal.core.executor.ThreadLocalDelegateCallable.call(ThreadLocalDelegateCallable.java:38)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.lang.Thread.run(Thread.java:748)
... 34 frames trimmed
Caused by: oracle.jdbc.OracleDatabaseException: ORA-01652: unable to extend temp segment by 128 in tablespace STASHTBS
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:498)
... 40 common frames omitted
2021-04-17 05:32:19,155 WARN [active-objects-init-compatibility-tenant-0] n.java.ao.db.OracleDatabaseProvider Error in schema creation: ORA-01652: unable to extend temp segment by 128 in tablespace STASHTBS
; attempting to roll back last partially generated table
2021-04-17 05:32:19,188 WARN [active-objects-init-compatibility-tenant-0] c.a.a.osgi.TenantAwareActiveObjects bundle [com.atlassian.audit.atlassian-audit-plugin] failed to create ActiveObjects
com.atlassian.activeobjects.internal.ActiveObjectsSqlException: There was a SQL exception thrown by the Active Objects library:
Database:
- name:Oracle
- version:Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
- minor version:1
- major version:12
Driver:
- name:Oracle JDBC driver
- version:12.2.0.1.0
java.sql.SQLException: ORA-01652: unable to extend temp segment by 128 in tablespace STASHTBS
at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.migrate(EntityManagedActiveObjects.java:54)
at com.atlassian.activeobjects.internal.AbstractActiveObjectsFactory.lambda$create$0(AbstractActiveObjectsFactory.java:73)
at com.atlassian.sal.core.transaction.HostContextTransactionTemplate$1.doInTransaction(HostContextTransactionTemplate.java:21)
at com.atlassian.stash.internal.sal.spi.HostContextAccessorImpl.doInTransaction(HostContextAccessorImpl.java:54)
at com.atlassian.sal.core.transaction.HostContextTransactionTemplate.execute(HostContextTransactionTemplate.java:18)
at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
at com.atlassian.activeobjects.internal.AbstractActiveObjectsFactory.create(AbstractActiveObjectsFactory.java:70)
at com.atlassian.activeobjects.internal.DelegatingActiveObjectsFactory.create(DelegatingActiveObjectsFactory.java:32)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:91)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:86)
at com.atlassian.sal.core.executor.ThreadLocalDelegateCallable.call(ThreadLocalDelegateCallable.java:38)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.lang.Thread.run(Thread.java:748)
... 34 frames trimmed
Caused by: java.sql.SQLException: ORA-01652: unable to extend temp segment by 128 in tablespace STASHTBS
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:494)
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:446)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1054)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:623)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:213)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:37)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:896)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1119)
at oracle.jdbc.driver.OracleStatement.executeUpdateInternal(OracleStatement.java:1661)
at oracle.jdbc.driver.OracleStatement.executeLargeUpdate(OracleStatement.java:1626)
at oracle.jdbc.driver.OracleStatement.executeUpdate(OracleStatement.java:1613)
at oracle.jdbc.driver.OracleStatementWrapper.executeUpdate(OracleStatementWrapper.java:282)
at com.zaxxer.hikari.pool.ProxyStatement.executeUpdate(ProxyStatement.java:120)
at com.zaxxer.hikari.pool.HikariProxyStatement.executeUpdate(HikariProxyStatement.java)
at net.java.ao.DatabaseProvider.executeUpdate(DatabaseProvider.java:2354)
at net.java.ao.DatabaseProvider.executeUpdateForAction(DatabaseProvider.java:2410)
at net.java.ao.DatabaseProvider.executeUpdatesForActions(DatabaseProvider.java:2382)
at net.java.ao.schema.SchemaGenerator.migrate(SchemaGenerator.java:91)
at net.java.ao.EntityManager.migrate(EntityManager.java:131)
at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.migrate(EntityManagedActiveObjects.java:52)
... 19 common frames omitted
Caused by: oracle.jdbc.OracleDatabaseException: ORA-01652: unable to extend temp segment by 128 in tablespace STASHTBS
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:498)
... 40 common frames omitted
Audit logs cannot be displayed due to the error "Failed to get action filters":
2021-04-17 05:51:21,134 ERROR [https-jsse-nio-8443-exec-5] sbhandiwad @7I3MC2x351x1084x1 n9eru0 10.80.201.103 "GET /plugins/servlet/audit/undefined/rest/shortcuts/latest/shortcuts/738dc90/bundled HTTP/1.1" c.a.a.f.d.AuditEventsViewDataProvider Failed to get action filters
com.atlassian.activeobjects.internal.ActiveObjectsInitException: bundle [com.atlassian.audit.atlassian-audit-plugin]
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:95)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:86)
at com.atlassian.sal.core.executor.ThreadLocalDelegateCallable.call(ThreadLocalDelegateCallable.java:38)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.lang.Thread.run(Thread.java:748)
... 1 frame trimmed
Caused by: com.atlassian.activeobjects.internal.ActiveObjectsSqlException: There was a SQL exception thrown by the Active Objects library:
Database:
- name:Oracle
- version:Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
- minor version:1
- major version:12
Driver:
- name:Oracle JDBC driver
- version:12.2.0.1.0
java.sql.SQLException: ORA-01652: unable to extend temp segment by 128 in tablespace STASHTBS
Cause
This issue is caused by the lack of free space in Bitbucket's Oracle database.
Solution
To resolve this issue java.io.IOException: ORA-01652: unable to extend temp segment