如果zookeeper leader进程被杀，所有follower都应该得到异常并重新启动吗？

Question

我正在使用 Zookeeper 3.4.6 进行一个项目，并且正在执行一些故障模式测试。这样做时，我发现了（我认为是）意外的行为。

如果领导者 Zookeeper 进程被杀死，追随者是否应该重新启动？

环境：

OS:        Windows Server 2008 R2 (hosted in a Tanuki Java service wrapper)
Zookeeper: 3.4.6
Java JDK:  1.7.0.210

测试：

测试是杀死 Zookeeper 进程并确保集群恢复。

如果我杀死一个非领导进程，它会重新启动并重新加入集群，而不会影响其他节点。

如果我杀死领导进程，领导和追随者将重新启动。这似乎不对，因为有一段时间客户端无法连接到任何Zookeeper 节点。

我已经尝试了TCP和UDP通信设置，但都表现出相同的行为。不过，UDP 的恢复速度是后者的两倍。

动物园管理员设置

tickTime=2000
initLimit=5
syncLimit=2
minSessionTimeout=5000
maxSessionTimeout=120000
dataDir=C:\\ProgramData\\Saab OneView\\ZooKeeper\\zoo-data
clientPort=2181
leaderServes=yes
autopurge.purgeInterval=24

# IP addresses blanked out here
server.1=0.0.0.1:2888:3888
server.2=0.0.0.2:2888:3888
server.3=0.0.0.3:2888:3888
server.4=0.0.0.4:2888:3888
server.5=0.0.0.5:2888:3888

# This is for zookeeper->zookeeper communication
# I've tried both settings, UDP has faster recovery time
# 0 = UDP 
# 3 = TCP (default)
electionAlg=3

导致关闭的示例跟随器异常

20160309 05:35:51.958Z 20160309 05:35:51.958 [myid:3] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@780] - Connection broken for id 4, my id = 3, error = 
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.io.DataInputStream.readInt(Unknown Source)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:765)
20160309 05:35:51.959Z 20160309 05:35:51.959 [myid:3] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@783] - Interrupting SendWorker
20160309 05:35:51.959Z 20160309 05:35:51.959 [myid:3] - WARN  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Follower@89] - Exception when following the leader
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.io.BufferedInputStream.fill(Unknown Source)
    at java.io.BufferedInputStream.read(Unknown Source)
    at java.io.DataInputStream.readInt(Unknown Source)
    at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
    at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83)
    at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103)
    at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:153)
    at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:85)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:786)
20160309 05:35:51.960Z 20160309 05:35:51.960 [myid:3] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:2181:Follower@166] - shutdown called
java.lang.Exception: shutdown Follower
    at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:790)

Answer 1

根据ZOOKEEPER-3478，这是预期行为：

在领导者选举期间，所有追随者关闭是正常行为。由于领导者崩溃后不再有领导者，因此曾经是追随者的服务器不再是追随者。因此，追随者关闭并返回“LOOKING”状态以找到新的领导者。