wurstmeister 的 Kafka-docker 容器扩展失败，错误是“代理 1001 已经注册了广告侦听器”

Question

单个 Kafka-docker 代理实例与下载的 wurstmeister 映像一起工作正常。执行以下命令结果在 cmd 提示符下显示为已完成：

docker-compose scale kafka=2
WARNING: The scale command is deprecated. Use the up command with the --scale flag instead.
Starting mskafka_kafka_1 ... done
Creating mskafka_kafka_2 ... done

虽然，容器的状态是 Exit 1

docker-compose ps
The system cannot find the path specified.
       Name                      Command               State                          Ports
------------------------------------------------------------------------------------------------------------------
mskafka_apache_1      /bin/sh -c apache2ctl -D F ...   Up       0.0.0.0:8080->80/tcp
mskafka_kafka_1       start-kafka.sh                   Up       0.0.0.0:9092->9093/tcp
mskafka_kafka_2       start-kafka.sh                   Exit 1
mskafka_postgres_1    docker-entrypoint.sh postgres    Up       5432/tcp
mskafka_zookeeper_1   /bin/sh -c /usr/sbin/sshd  ...   Up       0.0.0.0:2181->2181/tcp, 22/tcp, 2888/tcp, 3888/tcp

Configuration file : docker-compose.yml

kafka:
    image: wurstmeister/kafka:2.11-2.0.0
    links:
     - zookeeper
    environment:
      KAFKA_ADVERTISED_HOST_NAME: kafka
     # KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_CREATE_TOPICS: "order:5:1"
    ports:
     - "9092"

容器日志显示以下堆栈跟踪：

[2018-12-27 11:12:42,035] INFO [LogDirFailureHandler]: Starting (kafka.server.ReplicaManager$LogDirFailureHandler)
[2018-12-27 11:12:42,056] ERROR [KafkaServer id=1002] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.IllegalArgumentException: requirement failed: Configured end points kafka:9092 in advertised listeners are already registered by broker 1001
        at scala.Predef$.require(Predef.scala:224)
        at kafka.server.KafkaServer$$anonfun$createBrokerInfo$2.apply(KafkaServer.scala:384)
        at kafka.server.KafkaServer$$anonfun$createBrokerInfo$2.apply(KafkaServer.scala:382)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
        at kafka.server.KafkaServer.createBrokerInfo(KafkaServer.scala:382)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:256)
        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
        at kafka.Kafka$.main(Kafka.scala:75)
        at kafka.Kafka.main(Kafka.scala)
[2018-12-27 11:12:42,065] INFO [KafkaServer id=1002] shutting down (kafka.server.KafkaServer)

环境详细信息：1. 操作系统 - Windows 10 2. Docker kafka 镜像：wurstmeister/kafka:2.11-2.0.0

是否缺少任何配置更改？

Answer 1

您不能像扩展某个 Web 应用程序或 API 那样扩展 Kafka 代理。

至少这些属性对于每个经纪人都需要是唯一的

• KAFKA_BROKER_ID （这已经为你处理了）
• KAFKA_ADVERTISED_HOST_NAME并KAFKA_ADVERTISED_PORT知道代理存在于网络上的哪个位置（这些已弃用，应改用广告中的侦听器配置）
• KAFKA_ADVERTISED_LISTENERS是客户如何知道如何联系特定经纪人；如果允许多个代理相同，那么只有其中一个会收到任何生产者/消费者请求。该HOSTNAME_COMMAND可以用来设置这一点，但你必须为端口做一些特别的东西

注意内部topic的默认复制因子只有1，从1个broker扩展到3个broker不会改变这个

如果您查看 wurstmeister（和 confluentinc/cp-docker-images）存储库，它们具有多代理撰写文件，或者您可以考虑将 Strimzi / Bitnami Kubernetes 运算符用于 Kafka