Bitbucket Data Center shows "command timed out" error on creating a repository
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Summary:
While creating a repository on the Bitbucket Data Center GUI, an error message "command timed out" pops up after some time.
Environment
Tested on Bitbucket Data Center 8.9.3
Diagnosis
The
atlassian-bitbucket.logwill see the following logs:
1
2
3
4
5
6
2024-01-23 11:57:43,210 DEBUG [http-nio-7990-exec-10 url: /projects/PRO1/repos, /mvc/projects/PRO1/repos; user: user1] user1 @14HOEJJx715x161x0 ckqa1o 10.211.XX.X "POST /projects/PRO1/repos HTTP/1.1" c.a.s.i.r.DefaultRepositoryService Could not create repository PRO1/repo6
com.atlassian.bitbucket.scm.CommandTimeoutException: command timed out
... 400 frames trimmed
Caused by: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 129.987831398s. [closed=[], open=[[remote_addr=localhost/127.0.0.1:7777]]]
at io.grpc.Status.asRuntimeException(Status.java:535) Click here for the full stack trace
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
2024-01-23 11:57:43,210 DEBUG [http-nio-7990-exec-10 url: /projects/PRO1/repos, /mvc/projects/PRO1/repos; user: user1] user1 @14HOEJJx715x161x0 ckqa1o 10.211.XX.X "POST /projects/PRO1/repos HTTP/1.1" c.a.s.i.r.DefaultRepositoryService Could not create repository PRO1/repo6
com.atlassian.bitbucket.scm.CommandTimeoutException: command timed out
at com.atlassian.stash.internal.scm.git.mesh.DefaultErrorTranslator.translateIfKnownCause(DefaultErrorTranslator.java:277)
at com.atlassian.stash.internal.scm.git.mesh.DefaultErrorTranslator.maybeTranslate(DefaultErrorTranslator.java:57)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.maybeTranslate(AbstractFutureResponseObserver.java:209)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver$ResponseFuture.translateCause(AbstractFutureResponseObserver.java:298)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver$ResponseFuture.get(AbstractFutureResponseObserver.java:235)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:151)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:139)
at com.atlassian.stash.internal.scm.git.mesh.GrpcRepositoryClient.create(GrpcRepositoryClient.java:86)
at com.atlassian.stash.internal.scm.git.mesh.MeshGitCommandFactory$10.call(MeshGitCommandFactory.java:391)
at com.atlassian.stash.internal.scm.git.mesh.MeshGitCommandFactory$10.call(MeshGitCommandFactory.java:387)
at com.atlassian.stash.internal.scm.PluginScmService.create(PluginScmService.java:202)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.createRepositoryInScm(DefaultRepositoryService.java:1008)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.lambda$null$7(DefaultRepositoryService.java:1060)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.lambda$createRepositoryWithRetry$8(DefaultRepositoryService.java:1060)
at io.atlassian.fugue.retry.RetrySupplier.get(RetrySupplier.java:100)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.createRepositoryWithRetry(DefaultRepositoryService.java:1079)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.create(DefaultRepositoryService.java:214)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.create(DefaultRepositoryService.java:89)
at com.atlassian.stash.internal.web.admin.RepositoryAdminController.createRepository(RepositoryAdminController.java:175)
at com.atlassian.stash.internal.web.admin.RepositoryAdminController$$FastClassBySpringCGLIB$$1318105d.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at com.atlassian.stash.internal.web.admin.RepositoryAdminController$$EnhancerBySpringCGLIB$$362aaa8.createRepository(<generated>)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.bitbucket.internal.ratelimit.servlet.filter.RateLimitFilter.doFilter(RateLimitFilter.java:75)
at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.stash.internal.web.auth.AuthorizationFailureInterceptor.doFilterInternal(AuthorizationFailureInterceptor.java:39)
at com.atlassian.stash.internal.spring.security.StashAuthenticationFilter.doFilter(StashAuthenticationFilter.java:85)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doInsideSpringSecurityChain(BeforeLoginPluginAuthenticationFilter.java:112)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:75)
at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:94)
at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67)
at com.atlassian.oauth2.provider.core.web.AccessTokenFilter.doFilter(AccessTokenFilter.java:81)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.plugins.authentication.basicauth.filter.DisableBasicAuthFilter.doFilter(DisableBasicAuthFilter.java:70)
at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:94)
at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67)
at com.atlassian.oauth2.provider.core.web.AccessTokenFilter.doFilter(AccessTokenFilter.java:81)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.plugin.connect.plugin.auth.oauth2.DefaultSalAuthenticationFilter.doFilter(DefaultSalAuthenticationFilter.java:69)
at com.atlassian.plugin.connect.plugin.auth.user.ThreeLeggedAuthFilter.doFilter(ThreeLeggedAuthFilter.java:109)
at com.atlassian.plugins.authentication.basicauth.filter.DisableBasicAuthFilter.doFilter(DisableBasicAuthFilter.java:70)
at com.atlassian.jwt.internal.servlet.JwtAuthFilter.doFilter(JwtAuthFilter.java:37)
at com.atlassian.analytics.client.filter.DefaultAnalyticsFilter.doFilter(DefaultAnalyticsFilter.java:26)
at com.atlassian.analytics.client.filter.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:33)
at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doBeforeBeforeLoginFilters(BeforeLoginPluginAuthenticationFilter.java:90)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:73)
at com.atlassian.stash.internal.request.DefaultRequestManager.doAsRequest(DefaultRequestManager.java:84)
at com.atlassian.stash.internal.hazelcast.ConfigurableWebFilter.doFilter(ConfigurableWebFilter.java:38)
at java.lang.Thread.run(Thread.java:750)
... 400 frames trimmed
Caused by: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 129.987831398s. [closed=[], open=[[remote_addr=localhost/127.0.0.1:7777]]]
at io.grpc.Status.asRuntimeException(Status.java:535)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:479)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.atlassian.stash.internal.scm.git.mesh.LastSeenClientInterceptor$LastSeenClientListener.onClose(LastSeenClientInterceptor.java:40)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.atlassian.stash.internal.scm.git.mesh.StatefulClientCallListener.onClose(StatefulClientCallListener.java:34)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.atlassian.stash.internal.scm.git.mesh.DeadlinePropagatingClientInterceptor$DeadlinePropagatingListener.onClose(DeadlinePropagatingClientInterceptor.java:156)
at com.atlassian.stash.internal.scm.git.mesh.ErrorHandlingClientInterceptor$ErrorHandlingCall$1.onClose(ErrorHandlingClientInterceptor.java:149)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
... 2 common frames omitted
2024-01-23 11:57:43,243 DEBUG [http-nio-7990-exec-10 url: /projects/PRO1/repos, /mvc/projects/PRO1/repos; user: user1] user1 @14HOEJJx715x161x0 ckqa1o 10.211.XX.X "POST /projects/PRO1/repos HTTP/1.1" c.a.s.i.w.a.RepositoryAdminController Exception occurred
com.atlassian.bitbucket.scm.CommandTimeoutException: command timed out
at com.atlassian.stash.internal.scm.git.mesh.DefaultErrorTranslator.translateIfKnownCause(DefaultErrorTranslator.java:277)
at com.atlassian.stash.internal.scm.git.mesh.DefaultErrorTranslator.maybeTranslate(DefaultErrorTranslator.java:57)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.maybeTranslate(AbstractFutureResponseObserver.java:209)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver$ResponseFuture.translateCause(AbstractFutureResponseObserver.java:298)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver$ResponseFuture.get(AbstractFutureResponseObserver.java:235)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:151)
at com.atlassian.stash.internal.scm.git.mesh.AbstractFutureResponseObserver.asResult(AbstractFutureResponseObserver.java:139)
at com.atlassian.stash.internal.scm.git.mesh.GrpcRepositoryClient.create(GrpcRepositoryClient.java:86)
at com.atlassian.stash.internal.scm.git.mesh.MeshGitCommandFactory$10.call(MeshGitCommandFactory.java:391)
at com.atlassian.stash.internal.scm.git.mesh.MeshGitCommandFactory$10.call(MeshGitCommandFactory.java:387)
at com.atlassian.stash.internal.scm.PluginScmService.create(PluginScmService.java:202)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.createRepositoryInScm(DefaultRepositoryService.java:1008)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.lambda$null$7(DefaultRepositoryService.java:1060)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.lambda$createRepositoryWithRetry$8(DefaultRepositoryService.java:1060)
at io.atlassian.fugue.retry.RetrySupplier.get(RetrySupplier.java:100)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.createRepositoryWithRetry(DefaultRepositoryService.java:1079)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.create(DefaultRepositoryService.java:214)
at com.atlassian.stash.internal.repository.DefaultRepositoryService.create(DefaultRepositoryService.java:89)
at com.atlassian.stash.internal.web.admin.RepositoryAdminController.createRepository(RepositoryAdminController.java:175)
at com.atlassian.stash.internal.web.admin.RepositoryAdminController$$FastClassBySpringCGLIB$$1318105d.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at com.atlassian.stash.internal.web.admin.RepositoryAdminController$$EnhancerBySpringCGLIB$$362aaa8.createRepository(<generated>)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.bitbucket.internal.ratelimit.servlet.filter.RateLimitFilter.doFilter(RateLimitFilter.java:75)
at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.stash.internal.web.auth.AuthorizationFailureInterceptor.doFilterInternal(AuthorizationFailureInterceptor.java:39)
at com.atlassian.stash.internal.spring.security.StashAuthenticationFilter.doFilter(StashAuthenticationFilter.java:85)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doInsideSpringSecurityChain(BeforeLoginPluginAuthenticationFilter.java:112)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:75)
at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:94)
at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67)
at com.atlassian.oauth2.provider.core.web.AccessTokenFilter.doFilter(AccessTokenFilter.java:81)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.plugins.authentication.basicauth.filter.DisableBasicAuthFilter.doFilter(DisableBasicAuthFilter.java:70)
at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:94)
at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67)
at com.atlassian.oauth2.provider.core.web.AccessTokenFilter.doFilter(AccessTokenFilter.java:81)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.plugin.connect.plugin.auth.oauth2.DefaultSalAuthenticationFilter.doFilter(DefaultSalAuthenticationFilter.java:69)
at com.atlassian.plugin.connect.plugin.auth.user.ThreeLeggedAuthFilter.doFilter(ThreeLeggedAuthFilter.java:109)
at com.atlassian.plugins.authentication.basicauth.filter.DisableBasicAuthFilter.doFilter(DisableBasicAuthFilter.java:70)
at com.atlassian.jwt.internal.servlet.JwtAuthFilter.doFilter(JwtAuthFilter.java:37)
at com.atlassian.analytics.client.filter.DefaultAnalyticsFilter.doFilter(DefaultAnalyticsFilter.java:26)
at com.atlassian.analytics.client.filter.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:33)
at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46)
at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doBeforeBeforeLoginFilters(BeforeLoginPluginAuthenticationFilter.java:90)
at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:73)
at com.atlassian.stash.internal.request.DefaultRequestManager.doAsRequest(DefaultRequestManager.java:84)
at com.atlassian.stash.internal.hazelcast.ConfigurableWebFilter.doFilter(ConfigurableWebFilter.java:38)
at java.lang.Thread.run(Thread.java:750)
... 400 frames trimmed
Caused by: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 129.987831398s. [closed=[], open=[[remote_addr=localhost/127.0.0.1:7777]]]
at io.grpc.Status.asRuntimeException(Status.java:535)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:479)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.atlassian.stash.internal.scm.git.mesh.LastSeenClientInterceptor$LastSeenClientListener.onClose(LastSeenClientInterceptor.java:40)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.atlassian.stash.internal.scm.git.mesh.StatefulClientCallListener.onClose(StatefulClientCallListener.java:34)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.atlassian.stash.internal.scm.git.mesh.DeadlinePropagatingClientInterceptor$DeadlinePropagatingListener.onClose(DeadlinePropagatingClientInterceptor.java:156)
at com.atlassian.stash.internal.scm.git.mesh.ErrorHandlingClientInterceptor$ErrorHandlingCall$1.onClose(ErrorHandlingClientInterceptor.java:149)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
... 2 common frames omittedTheatlassian-mesh.logwill have the following events:
1
2
3
4
5
6
2024-01-23 11:55:33,084 DEBUG [grpc-server:thread-9] user1 3CK63RGEx715x34x2 @14HOEJJx715x161x0 127.0.0.1 "RepositoryService/Create" (>1 <0) c.a.b.m.r.DefaultRpcRepositoryAccessTracker all has 1 active requests
2024-01-23 11:55:33,086 DEBUG [grpc-server:thread-9] user1 3CK63RGEx715x34x2 @14HOEJJx715x161x0 127.0.0.1 "RepositoryService/Create" (>1 <0) c.a.b.m.t.DefaultTransactionManager Registering transaction n/sidecar/tx/99c9dde90000000000000003 to automatically roll back on cleanup of the current RPC
2024-01-23 11:55:33,087 DEBUG [grpc-server:thread-9] user1 3CK63RGEx715x34x2 @14HOEJJx715x161x0 127.0.0.1 "RepositoryService/Create" (>1 <0) c.a.b.m.r.DefaultRepositoryManager Creating repository for ds/0/h/26272df14de3e8b0f131/r/2
2024-01-23 11:55:33,164 DEBUG [grpc-server:thread-9] user1 3CK63RGEx715x34x2 @14HOEJJx715x161x0 127.0.0.1 "RepositoryService/Create" (>1 <0) c.a.bitbucket.mesh.process.OsProcess 8562: [git init --bare --quiet --shared=false --template=/opt/homedir/shared/config/git/templates /opt/homedir/shared/data/repositories/2] exited 0 in 49ms (stdin: 0, stdout: 0, stderr: 0)
2024-01-23 11:55:33,165 DEBUG [grpc-server:thread-9] user1 3CK63RGEx715x34x2 @14HOEJJx715x161x0 127.0.0.1 "RepositoryService/Create" (>1 <0) c.a.b.m.r.DefaultRepositoryManager 26272df14de3e8b0f131-2: Configuring refs/heads/master as the default branch
2024-01-23 11:55:33,168 DEBUG [grpc-server:thread-9] user1 3CK63RGEx715x34x2 @14HOEJJx715x161x0 127.0.0.1 "RepositoryService/Create" (>1 <0) c.a.b.m.r.DefaultRepositoryManager Writing metadata for 26272df14de3e8b0f131-2Cause
The problem happens when the procedure of creating the bare repository on the filesystem fails because the Bitbucket server node (NFS client) is unable to communicate with the NFS server. Bitbucket waits by default for 130 seconds for the operation to finish, and if it doesn't, the GUI reports an error saying "command timed out".
The following two factors could be the cause of this problem.
The Bitbucket server and the NFS server do not have the necessary NFS ports whitelisted.
The lock mechanism in NFS isn't operating correctly. Git generates a lock file called
repository-config.lockinside the repository directory when the repository is created, which it later renames torepository-config.
In this particular case, the repository directory is created but git couldn't rename the repository-config.lock in the path <Bitbucket-home-directory>/shared/data/repositories/<repo-ID> into a regular file because the statd and lockd ports on the NFS server aren't reachable from the Bitbucket server. Checking the contents of the <Bitbucket-home-directory>/shared/data/repositories/<repo-ID>, you will see the below:
1
2
3
4
5
6
7
8
9
ls -ltrh <Bitbucket-home-directory>/shared/data/repositories/<repo-ID>
total 12K
drwxr-xr-x 1 atlbitbucket atlbitbucket 100 Jan 23 11:55 hooks
drwxr-xr-x 1 atlbitbucket atlbitbucket 18 Jan 23 11:55 refs
-rw-r--r-- 1 atlbitbucket atlbitbucket 548 Jan 23 11:55 config
drwxr-xr-x 1 atlbitbucket atlbitbucket 16 Jan 23 11:55 objects
-rw-r--r-- 2 atlbitbucket atlbitbucket 0 Jan 23 12:27 repository-config.lock
-rw-r--r-- 1 atlbitbucket atlbitbucket 23 Jan 23 11:55 HEADSolution
Check all the ports listed on the NFS servers are accessible from the Bitbucket server node(s). The following command can be used to list the port.
1rpcinfo -p | awk '{print $3" "$4}' | sort -k2n | uniqCheck the lock functionality of the NFS. It can be checked by running the below command inside the shared home folder on the Bitbucket server node. The command should successfully exit and the file
testshould be generated.1 2cd <Bitbucket-home-directory>/shared flock <Bitbucket-home-directory>/shared/test.lock mv test.lock testIf any of the above-mentioned requirements are not met then Bitbucket can't create the repository. In this situation, please seek help from the system admin or OS vendor to resolve the related problem.
Updated on February 27, 2025
Was this helpful?
Still need help?
The Atlassian Community is here for you.