生成消息时找不到主题:UNKNOWN_TOPIC_OR_PARTITION

Question

我有一个双节点kafka集群(EC2实例),其中每个节点都用作一个单独的代理.当我使用以下命令在leader实例上运行生产者时:

kafka-console-producer.sh   --broker-list localhost:9092 --topic test

我收到以下错误.

测试消息[2017-01-09 13:22:39,483] WARN在获取具有相关ID 0的元数据时出错:{test = UNKNOWN_TOPIC_OR_PARTITION}(org.apache.kafka.clients.NetworkClient)[2017-01-09 13:22: [39,562] WARN获取具有相关ID 1的元数据时出错:{test = UNKNOWN_TOPIC_OR_PARTITION}(org.apache.kafka.clients.NetworkClient)[2017-01-09 13:22:39,663] WARN获取具有相关ID为2的元数据时出错: {test = UNKNOWN_TOPIC_OR_PARTITION}(org.apache.kafka.clients.NetworkClient)...

列出主题并kaka-topics.sh显示主题存在的显示.

主题描述:

kafka-topics.sh --zookeeper localhost:2181 --describe --topic test

回报

 Topic:test PartitionCount:8    ReplicationFactor:1 Configs:
     Topic: test    Partition: 0    Leader: 1   Replicas: 1 Isr: 1
     Topic: test    Partition: 1    Leader: 2   Replicas: 2 Isr: 2
     Topic: test    Partition: 2    Leader: 1   Replicas: 1 Isr: 1
     Topic: test    Partition: 3    Leader: 2   Replicas: 2 Isr: 2
     Topic: test    Partition: 4    Leader: 1   Replicas: 1 Isr: 1
     Topic: test    Partition: 5    Leader: 2   Replicas: 2 Isr: 2
     Topic: test    Partition: 6    Leader: 1   Replicas: 1 Isr: 1
     Topic: test    Partition: 7    Leader: 2   Replicas: 2 Isr: 2

我正在使用kafka 0.10.1.1.

server.propertes 文件包含:

listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners=PLAINTEXT://0.0.0.0:9092
port=9092
host.name=kafka-node1(kafka-node1 for the second host)
advertised.host.name=kafka-node1(kafka-node2 for the second host)
advertised.port=9092

当我尝试从第二个主机生成消息时,我收到以下消息:

WARN在主题分区test-4上获得错误产生响应且相关ID为1,重试(剩下2次尝试).错误:NOT_LEADER_FOR_PARTITION(org.apache.kafka.clients.producer.internals.Sender)....

有人可以帮忙吗？谢谢.

Answer 1

将主题的复制因子设置为 2 test，您将不会再遇到此异常。由于您有 2 个代理集群，因此执行以下命令来生成消息：

kafka-console-producer.sh   --broker-list localhost:9092,localhost:9093 --topic test

这将向集群中的两个代理发送消息。