Kafka使用者无法使用引导服务器名称来使用消息

Question

我在使用bootstrap-server(即Kafka服务器)消费消息时遇到问题.知道为什么没有zookeeper就无法使用消息？

• 卡夫卡版本:kafka_2.11-1.0.0
• Zookeeper版本:kafka_2.11-1.0.0
• Zookeeper主机和端口:zkp02.mp.com:2181
• Kafka主机和港口:kfk03.mp.com:9092

制作一些消息:

[kfk03.mp.com ~]$ /bnsf/kafka/bin/kafka-console-producer.sh --broker-list kfk03.mp.com:9092 --topic test
>hi
>hi

如果我给出消费者,消费者无法使用消息–-bootstrap-server:

[kfk03.mp.com ~]$
/bnsf/kafka/bin/kafka-console-consumer.sh --bootstrap-server kfk03.mp.com:9092 --topic test --from-beginning

消费者能够在--zookeeper给出服务器而不是--bootstrap-server- 时使用消息:

[kfk03.mp.com ~]$ /bnsf/kafka/bin/kafka-console-consumer.sh --zookeeper zkp02.mp.com:2181 --topic test --from-beginning

Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].

{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
hi
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
hi
hi
uttam
hi
hi
hi
hello
hi
^CProcessed a total of 17 messages

Answer 1

在使用bootstrap-server参数从kafka消费消息时,连接通过kafka服务器而不是zookeeper发生.Kafka经纪人在__consumer_offsets主题中存储偏移细节.

检查主题列表中是否存在__consumer_offsets.如果它不存在,请检查kafka日志以找到原因.

我们遇到了类似的问题.在我们的例子中,由于以下错误,未创建__consumer_offsets:

ERROR [KafkaApi-1001] Number of alive brokers '1' does not meet the required replication factor '3' for the offsets topic (configured via 'offsets.topic.replication.factor').