Bamboo hangs when uploading artifacts from elastic agents
Symptoms
Following error message shows up in the logs after the "Artifact Copier" starts and the connection seems to shut down:
2013-12-24 11:29:03,323 INFO [1-BAM::Elastic Agent on i333344445555::Agent:pool-3-thread-1] [ExecuteBuildTask] Running post build plugin 'Artifact Copier'
2013-12-24 11:29:03,334 INFO [1-BAM::Elastic Agent on i333344445555::Agent:pool-3-thread-1] [BuildArtifactPostProcessor] Copying the build artifacts for build: D-S-JOB1-1
2013-12-24 11:29:03,405 INFO [1-BAM::Elastic Agent on i333344445555::Agent:pool-3-thread-1] [AbstractArtifactManager] Publishing [processor-server.jar] for D-S-JOB1-1: 1 file(s) matching [processor-server.jar] in directory /home/bamboo/bamboo-agent-home/xml-data/build-dir/D-S-JOB1
2013-12-24 11:29:03,751 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-5] Accepted a local connection on port 46593, tunnel age: 199s
2013-12-24 11:29:49,481 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-11] Accepted a local connection on port 4527, tunnel age: 149s
2013-12-24 11:30:49,472 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-5] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:31:49,471 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-15] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:32:49,474 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-16] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:33:49,471 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-15] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:34:49,476 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-14] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:35:49,475 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-15] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:36:49,470 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-17] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:37:49,489 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-15] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:38:49,501 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-16] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:39:49,484 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-15] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:40:49,491 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-18] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:41:49,488 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-15] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:42:49,482 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-17] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:43:49,469 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-15] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:44:49,471 INFO [tunnellogger-thread] [LocalAcceptor] [tunnelserver:26224-1-thread-19] Accepted a local connection on port 4527, tunnel age: 59s
2013-12-24 11:45:33,164 ERROR [1-BAM::Elastic Agent on i333344445555::Agent:pool-1-thread-1] [HttpMessageSender] Exception occurred when posting message com.atlassian.bamboo.v2.build.agent.messages.ArtifactPublishMessage@63358yf8. Connection has been shutdown: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1358)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1370)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:44)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
....
at com.atlassian.bamboo.build.pipeline.concurrent.NamedThreadFactory$2.run(NamedThreadFactory.java:55)
at java.lang.Thread.run(Thread.java:662)
Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:190)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1764)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1725)
....
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
... 23 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
...
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:100)
... 44 moreCause
One root cause for this problem can be an old proxy server or firewall that is not supporting HTTP v1.1. Restart Bamboo and the elastic agent, and disable or bypass the firewall/proxy to test for this.
Resolution
Run an upgrade to have chunked requests supported. As an example, you will experience this problem if you are running nginx 1.1.9 (as it ships with Ubuntu LTS 12.04) and upgrading to 1.3.* using their repo (as that supports chunked requests natively) will fix this problem.
Last modified on Dec 26, 2013
Powered by Confluence and Scroll Viewport.