Bitbucket Data Center Migration export failed with command timed out exception
Platform Notice: Data Center - This article applies to Atlassian products on the Data Center platform.
Note that this knowledge base article was created for the Data Center version of the product. Data Center knowledge base articles for non-Data Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
The Bitbucket Data Center Migration export job is failing consistently on a specific repository with a "com.atlassian.bitbucket.scm.CommandTimeoutException: command timed out" message.
Environment
Bitbucket Data Center.
Diagnosis
The logs throw an error similar to the below lines:
2023-03-17 19:00:00,666 ERROR [dc-migration:thread-1] administrator *8Z6XYYx1069x5613034x0 www9ww 1.2.3.4 "POST /rest/api/1.0/migration/exports HTTP/1.1" c.a.s.i.m.DefaultExportContext Error registered for job 262 of type com.atlassian.bitbucket.migration.export (RUNNING) against repo: proj/repoSlug[9876]: Exception executing callback 'export' on 'class com.atlassian.stash.internal.scm.git.migration.GitRepositoryExporter': 'command timed out'
com.atlassian.bitbucket.scm.CommandTimeoutException: command timed out
at com.atlassian.stash.internal.scm.git.mesh.DefaultErrorTranslator.translateIfKnownCause(DefaultErrorTranslator.java:276)
at com.atlassian.stash.internal.scm.git.mesh.DefaultErrorTranslator.maybeTranslate(DefaultErrorTranslator.java:56)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.maybeTranslate(AbstractFutureResponseObserver.java:209)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver$ResponseFuture.translateCause(AbstractFutureResponseObserver.java:298)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver$ResponseFuture.get(AbstractFutureResponseObserver.java:235)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:151)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:139)
at com.atlassian.stash.internal.scm.git.mesh.GrpcRepositoryClient.setGarbageCollectionPaused(GrpcRepositoryClient.java:268)
..
Caused by: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 59.997777777s. [closed=[], open=[[remote_addr=localhost/127.0.0.1:7777]]]
at io.grpc.Status.asRuntimeException(Status.java:535)Cause
The export job process is waiting for the garbage collection on the repository to complete before it could start exporting. When the timeout is exceeded, the job will fail.
Solution
Check if the repository has stale locks in the filesystem. The lock is located under the app-info folder
ls -ltr $BITBUCKET_HOME/shared/data/repositories/<repo-id>/app-info- Inspect the files
gc.log.lockandgc.pid. If these files exist, garbage collection will not run because it indicates that garbage collection is currently running. - Check if the timestamp of these files is way past the current date. If they are, proceed to delete these files.
- Stop Bitbucket
- Take a backup of both the database and the filesystem
- Check that there are no longer any git processes running
- Delete
gc.log.lockandgc.pid - Start Bitbucket
- Run the export job again.