Jenkins Slave 在构建期间离线

Question

一台 Jenkins 1.651 master 和 slave 已经安装在 Windows Server 2012 上。 slave 专用于运行 Selenium 测试，但大部分时间它会离线：

Build step 'Execute Windows batch command' marked build as failure
ERROR: slave-tester is offline; cannot locate jdk7

和

Apr 20, 2016 2:20:05 PM INFO jenkins.slaves.DefaultJnlpSlaveReceiver handle
Disconnecting slave-tester as we are reconnected from the current peer
Apr 20, 2016 2:20:17 PM WARNING org.jenkinsci.remoting.nio.NioChannelHub run
Communication problem
java.io.IOException: An existing connection was forcibly closed by the remote host
    at sun.nio.ch.SocketDispatcher.read0(Native Method)
    at sun.nio.ch.SocketDispatcher.read(Unknown Source)
    at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
    at sun.nio.ch.IOUtil.read(Unknown Source)
    at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
    at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:136)
    at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:306)
    at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

Apr 20, 2016 2:20:17 PM WARNING jenkins.slaves.JnlpSlaveAgentProtocol$Handler$1 onClosed
NioChannelHub keys=2 gen=88003: Computer.threadPoolForRemoting [#2] for slave-tester terminated
java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@17d0b4b[name=xxx]
    at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:208)
    at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:628)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: An existing connection was forcibly closed by the remote host
    at sun.nio.ch.SocketDispatcher.read0(Native Method)
    at sun.nio.ch.SocketDispatcher.read(Unknown Source)
    at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
    at sun.nio.ch.IOUtil.read(Unknown Source)
    at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
    at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:136)
    at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:306)
    at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:561)
    ... 6 more

奴隶日志：

<===[JENKINS REMOTING CAPACITY]===>Slave.jar version: 2.53.3
Slave.jar version: 2.53.3
This is a Windows slave
This is a Windows slave
Connection terminated
Connection terminated
ERROR: Failed to install restarter
hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown
    at hudson.remoting.Request.abort(Request.java:297)
    at hudson.remoting.Channel.terminate(Channel.java:847)
    at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1080)
    at hudson.remoting.Channel$1.handle(Channel.java:501)
    at hudson.remoting.AbstractByteArrayCommandTransport$1.handle(AbstractByteArrayCommandTransport.java:61)
    at org.jenkinsci.remoting.nio.NioChannelHub$2.run(NioChannelHub.java:594)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
    at ......remote call to slave-tester(Native Method)
    at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
    at hudson.remoting.Request.call(Request.java:172)
    at hudson.remoting.Channel.call(Channel.java:780)
    at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller.install(JnlpSlaveRestarterInstaller.java:52)
    at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller.access$000(JnlpSlaveRestarterInstaller.java:33)
    at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$1.call(JnlpSlaveRestarterInstaller.java:39)
    at jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$1.call(JnlpSlaveRestarterInstaller.java:36)
    at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: hudson.remoting.Channel$OrderlyShutdown
    at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1080)
    at hudson.remoting.Channel$1.handle(Channel.java:501)
    at hudson.remoting.AbstractByteArrayCommandTransport$1.handle(AbstractByteArrayCommandTransport.java:61)
    at org.jenkinsci.remoting.nio.NioChannelHub$2.run(NioChannelHub.java:594)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    ... 4 more
Caused by: Command close created at
    at hudson.remoting.Command.<init>(Command.java:56)
    at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1074)
    at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:1072)
    at hudson.remoting.Channel.close(Channel.java:1156)
    at hudson.remoting.Channel.close(Channel.java:1138)
    at hudson.remoting.Channel$CloseCommand.execute(Channel.java:1079)
    at hudson.remoting.Channel$1.handle(Channel.java:501)
    at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60)

是什么导致了这个问题以及如何解决它？

Answer 1

类似问题

• https://issues.jenkins-ci.org/browse/JENKINS-27251
• https://issues.jenkins-ci.org/browse/JENKINS-6817
• https://issues.jenkins-ci.org/browse/JENKINS-28155
• https://issues.jenkins-ci.org/browse/JENKINS-28826
• https://issues.jenkins-ci.org/browse/JENKINS-22932
• https://bugreports.qt.io/browse/QTQAINFRA-928
• /sf/ask/2572073591/
• https://mailman.owncloud.org/pipermail/jenkins/2015-July/006091.html
• https://mailman.boum.org/pipermail/tails-dev/2014-December/007732.htm

可能的解决方案

基于此来源，部分堆栈跟踪：

at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

在办公室遇到同样的问题，将jdk7升级到最新的oracle-jdk8可以解决该问题。

如果问题仍然存在：

1. 请准确指出问题发生的时间，以便可以重现该问题
2. 请添加导致问题的构建的 Jenkins 构建日志。也许是由导致问题的作业执行的特定任务。
3. 构建作业时查看 Windows 任务管理器
4. 请添加成功构建之一的日志和失败构建之一的日志