Duplicate key value errors in logs in Bitbucket Server using PostgreSQL
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Problem
When performing certain actions in Bitbucket Server e.g. creating a project or repository. Errors such as the following may occur in the atlassian-bitbucket.log file
org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "AO_616D7B_BRANCH_MODEL_CONFIG_pkey"
Detail: Key ("ID")=(18) already exists.
at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.create(EntityManagedActiveObjects.java:102)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.create(TenantAwareActiveObjects.java:271)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
at com.atlassian.bitbucket.internal.branch.model.dao.AoBranchModelDao.create(AoBranchModelDao.java:49)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService.lambda$update$8(DefaultBranchModelService.java:337)
at java.util.Optional.orElseGet(Optional.java:267)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService.update(DefaultBranchModelService.java:336)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService.createDefaultConfiguration(DefaultBranchModelService.java:221)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService.access$300(DefaultBranchModelService.java:48)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService$3.visit(DefaultBranchModelService.java:274)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService$3.visit(DefaultBranchModelService.java:269)
at com.atlassian.stash.internal.project.InternalNormalProject.accept(InternalNormalProject.java:37)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService.lambda$getOrCreateProjectConfiguration$6(DefaultBranchModelService.java:269)
at java.util.Optional.orElseGet(Optional.java:267)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService.getOrCreateProjectConfiguration(DefaultBranchModelService.java:269)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService.lambda$null$2(DefaultBranchModelService.java:151)
at java.util.Optional.orElseGet(Optional.java:267)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService.lambda$getModel$3(DefaultBranchModelService.java:151)
at com.atlassian.sal.core.transaction.HostContextTransactionTemplate$1.doInTransaction(HostContextTransactionTemplate.java:21)
at com.atlassian.stash.internal.sal.spi.HostContextAccessorImpl.doInTransaction(HostContextAccessorImpl.java:54)
at com.atlassian.sal.core.transaction.HostContextTransactionTemplate.execute(HostContextTransactionTemplate.java:18)
at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
at com.atlassian.activeobjects.internal.SalTransactionManager.inTransaction(SalTransactionManager.java:42)
at com.atlassian.activeobjects.internal.AbstractLoggingTransactionManager.doInTransaction(AbstractLoggingTransactionManager.java:16)
at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.executeInTransaction(EntityManagedActiveObjects.java:204)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.executeInTransaction(TenantAwareActiveObjects.java:336)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
at com.atlassian.bitbucket.internal.branch.model.DefaultBranchModelService.getModel(DefaultBranchModelService.java:149)
at com.atlassian.bitbucket.internal.branch.analytics.DefaultAnalyticsBranchClassifier.classifyRef(DefaultAnalyticsBranchClassifier.java:26)
at com.atlassian.bitbucket.internal.branch.analytics.AnalyticsBranchTypeListener.onRepositoryPushEvent(AnalyticsBranchTypeListener.java:52)
at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:40)
... 9 common frames omitted
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "AO_616D7B_BRANCH_MODEL_CONFIG_pkey"
Detail: Key ("ID")=(18) already exists.
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:120)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
at net.java.ao.db.PostgreSQLDatabaseProvider.executeInsertReturningKey(PostgreSQLDatabaseProvider.java:336)
at net.java.ao.DatabaseProvider.insertReturningKey(DatabaseProvider.java:1868)
at net.java.ao.db.PostgreSQLDatabaseProvider.insertReturningKey(PostgreSQLDatabaseProvider.java:298)
at net.java.ao.EntityManager.create(EntityManager.java:366)
at net.java.ao.EntityManager.create(EntityManager.java:399)
at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.create(EntityManagedActiveObjects.java:100)
... 52 common frames omitted
Diagnosis
Environment
- Bitbucket Server is connected to a PSQL database
- You may have migrated or restored from a backup recently.
Diagnostic Steps
Check the max id currently used in the AO table mentioned in the error e.g AO_616D7B_BRANCH_MODEL_CONFIG
select max("ID") from "AO_616D7B_BRANCH_MODEL_CONFIG";
Check the Detail: Key ("ID")=(18) line to see what should be inside max(" "). If the message is Detail: Key ("AUDIT_ITEM_ID")=(18), the query would be select max("AUDIT_ITEM_ID") ...
Check table sequence:
select * from "AO_616D7B_BRANCH_MODEL_CONFIG_ID_seq";
Similar to the above, the "..ID_seq" portion is changed based on the actual key name. so something like AUDIT_ITEM_ID would turn the table into "..._AUDIT_ITEM_ID_seq"
If the last_value column returned in second query is lower than max ID returned in first query. This KB applies.
Cause
This is caused by broken Active Object table sequences due to the bug BSERV-12075 - Getting issue details... STATUS
Resolution
You can run through the steps in the PSQL Fixing Sequences guide and if that doesn't work you can run through the following:
You can fix the sequence for each affected table mentioned in the logs with the following SQL query:
- Run the first 2 SQL queries in the Diagnostic section above
- Run the following SQL query to reset the affected table sequence:
ALTER SEQUENCE "<table_name>_<ID Key name>_seq" RESTART WITH <max ID from first SQL query in diagnostic steps>;
e.g AO_616D7B_BRANCH_MODEL_CONFIG:
ALTER SEQUENCE "AO_616D7B_BRANCH_MODEL_CONFIG_ID_seq" RESTART WITH <max ID from first SQL query in diagnostic steps>;
e.g for a table that doesn't have "ID" as the key name such as the AO_BD73C3_PROJECT_AUDIT table:
ALTER SEQUENCE "AO_BD73C3_PROJECT_AUDIT_AUDIT_ITEM_ID_seq" RESTART WITH <max ID from first SQL query in diagnostic steps>;
Always back up your data before performing any modifications to the database. If possible, test any alter, insert, update, or delete SQL commands on a staging server first.
- Restart Bitbucket Server for the changes to take effect.
- Double-check any log files to ensure that any table affected by this has been resolved.