Stash backup fails - Read timed out

Symptoms

Stash backup fails:

  • The Backup Client times out and fails gracefully.
  • The DIY method freezes the instance and the progress must be updated to "100" before the  backup can be cancelled.

Backup Client

The following appears in the atlassian-sbc-YYYY-MM-DD.log:

2014-06-25 08:02:14,400 INFO  [main] c.a.s.i.b.client.RestStashService Starting database backup on Stash. It may be cancelled with token: 52ee07d66403e54a2b8cdebfc2fbff46f6a6c2f1
2014-06-25 08:02:14,403 DEBUG [main] c.a.s.i.b.c.FixedRateProgressPublisher Progress publishing has started
2014-06-25 08:02:18,479 DEBUG [threadpool2] c.a.s.i.b.client.wink.LoggingHandler HTTP GET http://stash.atlassian:7990/mvc/maintenance, headers: [Accept:[application/json], User-Agent:[Wink Client v1.1.2], X-Atlassian-Maintenance-Token:[7b3e054a4e832a90d4e863501e7684b57a5c6d69]]
2014-06-25 08:02:18,503 DEBUG [threadpool2] c.a.s.i.b.client.wink.LoggingHandler -> 200 OK (482x52x0)
2014-06-25 08:02:18,506 DEBUG [threadpool2] c.a.s.i.b.c.s.h.WinkClientHttpResponse HTTP status code: 200, status text: "OK", headers: [Cache-Control:[no-cache], Content-Type:[application/json], Date:[Wed, 25 Jun 2014 13:02:18 GMT], Server:[Apache-Coyote/1.1], Set-Cookie:[JSESSIONID=943C3419F09CC3777A7F03D2B77AB789; Path=/; Secure; HttpOnly], Transfer-Encoding:[chunked], Vary:[Accept-Encoding], X-AREQUESTID:[482x52x0], X-Content-Type-Options:[nosniff], X-Frame-Options:[SAMEORIGIN], X-XSS-Protection:[1; mode=block]]
2014-06-25 08:02:18,552 INFO  [main] c.a.s.i.b.client.DefaultBackupClient Waiting for SCM operations to finish
2014-06-25 08:02:23,482 DEBUG [threadpool2] c.a.s.i.b.client.wink.LoggingHandler HTTP GET http://stash.atlassian:7990/mvc/maintenance, headers: [Accept:[application/json], User-Agent:[Wink Client v1.1.2], X-Atlassian-Maintenance-Token:[7b3e054a4e832a90d4e863501e7684b57a5c6d69]]
2014-06-25 08:02:28,577 INFO  [main] c.a.s.i.b.client.DefaultBackupClient Waiting for SCM operations to finish
2014-06-25 08:02:38,580 INFO  [main] c.a.s.i.b.client.DefaultBackupClient Waiting for SCM operations to finish
2014-06-25 08:02:48,583 INFO  [main] c.a.s.i.b.client.DefaultBackupClient Waiting for SCM operations to finish
2014-06-25 08:02:58,586 INFO  [main] c.a.s.i.b.client.DefaultBackupClient Waiting for SCM operations to finish
2014-06-25 08:03:08,590 INFO  [main] c.a.s.i.b.client.DefaultBackupClient Waiting for SCM operations to finish
2014-06-25 08:03:18,617 DEBUG [main] c.a.s.i.b.c.FixedRateProgressPublisher Progress publishing has stopped
2014-06-25 08:03:18,625 DEBUG [main] c.a.s.i.b.client.RestStashService Unhandled client exception encountered while Stash was performing a database backup
com.atlassian.stash.internal.backup.client.ScmDrainTimedOutException: Operations from one or more SCMs did not finish within the allotted timeout. To prevent corruption due to inconsistent state, the backup has been aborted. Please try backup up again when the system is under less load.
	at com.atlassian.stash.internal.backup.client.DefaultBackupClient.waitForSCMDraining(DefaultBackupClient.groovy:183) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.FixedRateProgressPublisher.publishWhileExecuting(FixedRateProgressPublisher.groovy:34) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.RestStashService.backup(RestStashService.groovy:64) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.RestStashService.lock(RestStashService.groovy:172) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.DefaultBackupClient.backup(DefaultBackupClient.groovy:49) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.SpringMain.run(SpringMain.groovy:25) [stash-backup-core-1.2.1.jar:na]
	at com.atlassian.stash.internal.backup.client.BackupMain.main(BackupMain.groovy:38) [stash-backup-client.jar:1.2.1]
	... 87 frames suppressed

 

The following appears in the atlassian-stash.log:

2014-06-25 08:03:18,625 DEBUG [main] c.a.s.i.b.client.RestStashService Unhandled client exception encountered while Stash was performing a database backup
com.atlassian.stash.internal.backup.client.ScmDrainTimedOutException: Operations from one or more SCMs did not finish within the allotted timeout. To prevent corruption due to inconsistent state, the backup has been aborted. Please try backup up again when the system is under less load.
	at com.atlassian.stash.internal.backup.client.DefaultBackupClient.waitForSCMDraining(DefaultBackupClient.groovy:183) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.FixedRateProgressPublisher.publishWhileExecuting(FixedRateProgressPublisher.groovy:34) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.RestStashService.backup(RestStashService.groovy:64) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.RestStashService.lock(RestStashService.groovy:172) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.DefaultBackupClient.backup(DefaultBackupClient.groovy:49) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.SpringMain.run(SpringMain.groovy:25) [stash-backup-core-1.2.1.jar:na]
	at com.atlassian.stash.internal.backup.client.BackupMain.main(BackupMain.groovy:38) [stash-backup-client.jar:1.2.1]
	... 87 frames suppressed
2014-06-25 08:03:18,674 INFO  [main] c.a.s.i.b.client.RestStashService Cancelling database backup with code: 52ee07d66403e54a2b8cdebfc2fbff46f6a6c2f1
2014-06-25 08:03:18,679 DEBUG [main] c.a.s.i.b.client.wink.LoggingHandler HTTP POST http://stash.atlassian:7990/mvc/maintenance?token=52ee07d66403e54a2b8cdebfc2fbff46f6a6c2f1, headers: [Accept:[application/json], Content-Length:[0], Content-Type:[application/json], User-Agent:[Wink Client v1.1.2], X-Atlassian-Maintenance-Token:[7b3e054a4e832a90d4e863501e7684b57a5c6d69]]
2014-06-25 08:03:23,542 DEBUG [threadpool2] c.a.s.i.b.client.wink.LoggingHandler HTTP GET http://stash.atlassian:7990/mvc/maintenance, headers: [Accept:[application/json], User-Agent:[Wink Client v1.1.2], X-Atlassian-Maintenance-Token:[7b3e054a4e832a90d4e863501e7684b57a5c6d69]]
2014-06-25 08:04:18,719 ERROR [main] c.a.s.i.b.client.RestStashService Failed to cancel database backup: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
org.apache.wink.client.ClientRuntimeException: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
	at org.apache.wink.client.internal.ResourceImpl.invoke(ResourceImpl.java:241) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.invoke(ResourceImpl.java:189) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.invokeNoException(ResourceImpl.java:181) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.post(ResourceImpl.java:323) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.withBody(WinkClientHttpRequest.groovy:109) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.withPostBody(WinkClientHttpRequest.groovy:102) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.run(WinkClientHttpRequest.groovy:88) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.executeInternal(WinkClientHttpRequest.groovy:61) ~[stash-backup-client.jar:1.2.1]
	at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:52) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:527) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:489) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at org.springframework.web.client.RestTemplate.postForEntity(RestTemplate.java:344) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at com.atlassian.stash.internal.backup.client.RestStashService.cancelBackup(RestStashService.groovy:89) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.RestStashService.backup(RestStashService.groovy:73) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.RestStashService.lock(RestStashService.groovy:172) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.DefaultBackupClient.backup(DefaultBackupClient.groovy:49) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.SpringMain.run(SpringMain.groovy:25) [stash-backup-core-1.2.1.jar:na]
	at com.atlassian.stash.internal.backup.client.BackupMain.main(BackupMain.groovy:38) [stash-backup-client.jar:1.2.1]
	... 66 frames suppressed
Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.handle(HttpURLConnectionHandler.java:57) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.wink.LoggingHandler.handle(LoggingHandler.groovy:27) ~[stash-backup-client.jar:1.2.1]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.wink.AcceptHeaderHandler.handle(AcceptHeaderHandler.groovy:35) ~[stash-backup-client.jar:1.2.1]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.wink.PreemptiveBasicAuthSecurityHandler.handle(PreemptiveBasicAuthSecurityHandler.groovy:31) ~[stash-backup-client.jar:1.2.1]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.invoke(ResourceImpl.java:228) ~[wink-client-1.4.jar:1.4]
	... 83 common frames omitted
	... 3 frames suppressed
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.7.0_51]
	at java.net.SocketInputStream.read(SocketInputStream.java:152) ~[na:1.7.0_51]
	at java.net.SocketInputStream.read(SocketInputStream.java:122) ~[na:1.7.0_51]
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) ~[na:1.7.0_51]
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275) ~[na:1.7.0_51]
	at java.io.BufferedInputStream.read(BufferedInputStream.java:334) ~[na:1.7.0_51]
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687) ~[na:1.7.0_51]
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633) ~[na:1.7.0_51]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) ~[na:1.7.0_51]
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468) ~[na:1.7.0_51]
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.createResponse(HttpURLConnectionHandler.java:180) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.processResponse(HttpURLConnectionHandler.java:160) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.handle(HttpURLConnectionHandler.java:55) ~[wink-client-1.4.jar:1.4]
	... 94 common frames omitted
2014-06-25 08:04:18,736 DEBUG [main] c.a.s.i.b.client.RestStashService Unhandled client exception encountered while locking Stash for backup
org.apache.wink.client.ClientRuntimeException: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
	at org.apache.wink.client.internal.ResourceImpl.invoke(ResourceImpl.java:241) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.invoke(ResourceImpl.java:189) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.invokeNoException(ResourceImpl.java:181) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.post(ResourceImpl.java:323) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.withBody(WinkClientHttpRequest.groovy:109) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.withPostBody(WinkClientHttpRequest.groovy:102) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.run(WinkClientHttpRequest.groovy:88) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.executeInternal(WinkClientHttpRequest.groovy:61) ~[stash-backup-client.jar:1.2.1]
	at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:52) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:527) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:489) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at org.springframework.web.client.RestTemplate.postForEntity(RestTemplate.java:344) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at com.atlassian.stash.internal.backup.client.RestStashService.cancelBackup(RestStashService.groovy:89) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.RestStashService.backup(RestStashService.groovy:73) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.RestStashService.lock(RestStashService.groovy:172) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.DefaultBackupClient.backup(DefaultBackupClient.groovy:49) [stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.SpringMain.run(SpringMain.groovy:25) [stash-backup-core-1.2.1.jar:na]
	at com.atlassian.stash.internal.backup.client.BackupMain.main(BackupMain.groovy:38) [stash-backup-client.jar:1.2.1]
	... 66 frames suppressed
Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.handle(HttpURLConnectionHandler.java:57) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.wink.LoggingHandler.handle(LoggingHandler.groovy:27) ~[stash-backup-client.jar:1.2.1]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.wink.AcceptHeaderHandler.handle(AcceptHeaderHandler.groovy:35) ~[stash-backup-client.jar:1.2.1]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.wink.PreemptiveBasicAuthSecurityHandler.handle(PreemptiveBasicAuthSecurityHandler.groovy:31) ~[stash-backup-client.jar:1.2.1]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.invoke(ResourceImpl.java:228) ~[wink-client-1.4.jar:1.4]
	... 83 common frames omitted
	... 3 frames suppressed
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.7.0_51]
	at java.net.SocketInputStream.read(SocketInputStream.java:152) ~[na:1.7.0_51]
	at java.net.SocketInputStream.read(SocketInputStream.java:122) ~[na:1.7.0_51]
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) ~[na:1.7.0_51]
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275) ~[na:1.7.0_51]
	at java.io.BufferedInputStream.read(BufferedInputStream.java:334) ~[na:1.7.0_51]
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687) ~[na:1.7.0_51]
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633) ~[na:1.7.0_51]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) ~[na:1.7.0_51]
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468) ~[na:1.7.0_51]
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.createResponse(HttpURLConnectionHandler.java:180) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.processResponse(HttpURLConnectionHandler.java:160) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.handle(HttpURLConnectionHandler.java:55) ~[wink-client-1.4.jar:1.4]
	... 94 common frames omitted
2014-06-25 08:04:18,748 INFO  [main] c.a.s.i.b.client.RestStashService Unlocking Stash using token: 7b3e054a4e832a90d4e863501e7684b57a5c6d69
2014-06-25 08:04:18,752 DEBUG [main] c.a.s.i.b.client.wink.LoggingHandler HTTP DELETE http://localhost:7990/mvc/maintenance/lock?token=7b3e054a4e832a90d4e863501e7684b57a5c6d69, headers: User-Agent:[Wink Client v1.1.2], X-Atlassian-Maintenance-Token:[7b3e054a4e832a90d4e863501e7684b57a5c6d69]]
2014-06-25 08:04:18,776 DEBUG [main] c.a.s.i.b.client.wink.LoggingHandler -> 200 OK (484x57x3)
2014-06-25 08:04:18,778 DEBUG [main] c.a.s.i.b.c.s.h.WinkClientHttpResponse HTTP status code: 200, status text: "OK", headers: [Cache-Control:[no-cache], Content-Length:[0], Date:[Wed, 25 Jun 2014 13:04:18 GMT], Server:[Apache-Coyote/1.1], Set-Cookie:[JSESSIONID=66706902E12EE9980C35FDA701B17292; Path=/; Secure; HttpOnly], X-AREQUESTID:[484x57x3], X-Content-Type-Options:[nosniff], X-Frame-Options:[SAMEORIGIN], X-XSS-Protection:[1; mode=block]]
2014-06-25 08:04:18,789 ERROR [main] c.a.s.i.backup.client.BackupMain A backup could not be created. Reason: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
org.apache.wink.client.ClientRuntimeException: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
	at org.apache.wink.client.internal.ResourceImpl.invoke(ResourceImpl.java:241) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.invoke(ResourceImpl.java:189) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.invokeNoException(ResourceImpl.java:181) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.post(ResourceImpl.java:323) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.withBody(WinkClientHttpRequest.groovy:109) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.withPostBody(WinkClientHttpRequest.groovy:102) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.run(WinkClientHttpRequest.groovy:88) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.http.WinkClientHttpRequest.executeInternal(WinkClientHttpRequest.groovy:61) ~[stash-backup-client.jar:1.2.1]
	at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:52) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:527) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:489) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at org.springframework.web.client.RestTemplate.postForEntity(RestTemplate.java:344) ~[spring-web-4.0.0.RELEASE.jar:4.0.0.RELEASE]
	at com.atlassian.stash.internal.backup.client.RestStashService.cancelBackup(RestStashService.groovy:89) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.RestStashService.backup(RestStashService.groovy:73) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.RestStashService.lock(RestStashService.groovy:172) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.DefaultBackupClient.backup(DefaultBackupClient.groovy:49) ~[stash-backup-client.jar:1.2.1]
	at com.atlassian.stash.internal.backup.client.spring.SpringMain.run(SpringMain.groovy:25) [stash-backup-core-1.2.1.jar:na]
	at com.atlassian.stash.internal.backup.client.BackupMain.main(BackupMain.groovy:38) [stash-backup-client.jar:1.2.1]
	... 66 frames suppressed
Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.handle(HttpURLConnectionHandler.java:57) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.wink.LoggingHandler.handle(LoggingHandler.groovy:27) ~[stash-backup-client.jar:1.2.1]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.wink.AcceptHeaderHandler.handle(AcceptHeaderHandler.groovy:35) ~[stash-backup-client.jar:1.2.1]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at com.atlassian.stash.internal.backup.client.wink.PreemptiveBasicAuthSecurityHandler.handle(PreemptiveBasicAuthSecurityHandler.groovy:31) ~[stash-backup-client.jar:1.2.1]
	at org.apache.wink.client.internal.handlers.HandlerContextImpl.doChain(HandlerContextImpl.java:52) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.ResourceImpl.invoke(ResourceImpl.java:228) ~[wink-client-1.4.jar:1.4]
	... 83 common frames omitted
	... 3 frames suppressed
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.7.0_51]
	at java.net.SocketInputStream.read(SocketInputStream.java:152) ~[na:1.7.0_51]
	at java.net.SocketInputStream.read(SocketInputStream.java:122) ~[na:1.7.0_51]
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) ~[na:1.7.0_51]
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:275) ~[na:1.7.0_51]
	at java.io.BufferedInputStream.read(BufferedInputStream.java:334) ~[na:1.7.0_51]
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687) ~[na:1.7.0_51]
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633) ~[na:1.7.0_51]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) ~[na:1.7.0_51]
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468) ~[na:1.7.0_51]
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.createResponse(HttpURLConnectionHandler.java:180) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.processResponse(HttpURLConnectionHandler.java:160) ~[wink-client-1.4.jar:1.4]
	at org.apache.wink.client.internal.handlers.HttpURLConnectionHandler.handle(HttpURLConnectionHandler.java:55) ~[wink-client-1.4.jar:1.4]
	... 94 common frames omitted

Diagnosis

The Announcer for Stash plugin (versions up to and including 1.5.5.2) is configured with a "check access" filter that accesses a subsystem that has to remain untouched during a backup.

Workaround

Disable the Announcer for Stash plugin.

Resolution

Upgrade the plugin to version 1.5.5.4 or disable it prior to initiating a backup.

Last modified on Nov 2, 2018

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.