CuratorFrameworkImpl - 后台异常不可重试或重试放弃

Question

Curator 框架版本 - 4.3.0，Zookeeper 版本 - 5.5.0

我们在 Kubernetes 上部署了 apache atlas，它使用 Zookeeper 从两个 atlas pod 中选出一个作为领导者。我们正在运行 3 个 Zookeeper Pod（3 个节点集群），其中一个 Pod 宕机应该不会造成任何问题。当一个 Zookeeper Pod 宕机时，Zookeeper 集群仍然是健康的，并且有一个 Zookeeper Leader 可用。我通过执行 Zookeeper pod 并检查 Zookeeper 状态来测试这一点。但是 curator 框架抛出以下错误 -

[main:] ~ Background exception was not retry-able or retry gave up (CuratorFrameworkImpl:685)
java.net.UnknownHostException: zookeeper-2.zookeeper-headless.atlas.svc.cluster.local: Name or service not known
    at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
    at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
    at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
    at java.net.InetAddress.getAllByName(InetAddress.java:1193)
    at java.net.InetAddress.getAllByName(InetAddress.java:1127)
    at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:61)
    at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)
    at org.apache.curator.utils.DefaultZookeeperFactory.newZooKeeper(DefaultZookeeperFactory.java:29)
    at org.apache.curator.framework.imps.CuratorFrameworkImpl$2.newZooKeeper(CuratorFrameworkImpl.java:196)
    at org.apache.curator.HandleHolder$1.getZooKeeper(HandleHolder.java:101)
    at org.apache.curator.HandleHolder.getZooKeeper(HandleHolder.java:57)
    at org.apache.curator.ConnectionState.reset(ConnectionState.java:201)
    at org.apache.curator.ConnectionState.start(ConnectionState.java:111)
    at org.apache.curator.CuratorZookeeperClient.start(CuratorZookeeperClient.java:214)
    at org.apache.curator.framework.imps.CuratorFrameworkImpl.start(CuratorFrameworkImpl.java:314)
    at org.apache.atlas.web.service.CuratorFactory.initializeCuratorFramework(CuratorFactory.java:88)
    at org.apache.atlas.web.service.CuratorFactory.<init>(CuratorFactory.java:78)
    at org.apache.atlas.web.service.CuratorFactory.<init>(CuratorFactory.java:73)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:142)
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:89)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1152)

ZookeeperConnectionString =“zookeeper-0.zookeeper-headless.atlas.svc.cluster.local：2181，zookeeper-1.zookeeper-headless.atlas.svc.cluster.local：2181，zookeeper-2.zookeeper-headless.atlas.svc .cluster.local：2181”

我们面临的问题是，当我们尝试运行leaderLatch.start()时，它不会返回任何错误，但相应的znode没有在zookeeper中创建。

Answer 1

您在 Kubernetes 上看到错误的原因是，当 pod 重新启动时，其 DNS 记录也会在短时间内被删除，直到 pod 再次启动。在您的情况下，不会出现问题，因为 curator 将连接到您 CS 中的另一个 ZK 服务器。