标签: kcat

我正在尝试在 Windows 机器上安装 kafkaCat 工具。通过运行 ./bootstrap.sh 进行快速构建时，我收到以下错误：

$ ./bootstrap.sh
Directory librdkafka already exists, not downloading https://github.com/edenhill/librdkafka/archive/master.tar.gz
Directory libyajl already exists, not downloading https://github.com/lloyd/yajl/archive/master.tar.gz
Building librdkafka
checking for OS or distribution... ok (MINGW64_NT-6.1)
checking for C compiler from CC env... failed
checking for gcc (by command)... failed
checking for clang (by command)... failed
checking for cc (by command)... failed (fail)
checking for C++ compiler from CXX env... failed
checking for C++ compiler (g++)... failed
checking for C++ compiler (clang++)... failed
checking …

我需要向 Kafka 生成批量消息，因此我有一个向 kafkacat 提供的文件：

kafkacat -b localhost:9092 -t <my_topic> -T -P -l /tmp/msgs

/tmp/msgs内容如下

-H "id=1"
{"key" : "value0"}
-H "id=2"
{"key" : "value1"}

当我运行上面的 kafkacat 命令时，它会向 kafka 插入四条消息 - /tmp/msgs 中每行一条消息。

我需要指示 kafkacat 正确解析文件 - 即 -H "id=1" 是消息 {"key" = "value0"} 的标头。

我该如何实现这一目标？

谢谢

我在本地运行 Kafka。当我尝试写入 Kafka 时，出现以下错误：

kafkacat -b localhost:9092 -t req -T -P -l  msgs
hello
world
% ERROR: Local: Broker transport failure: localhost:9092/bootstrap: Connect to ipv6#[::1]:9092 failed: Connection refused (after 1ms in state CONNECT)
% ERROR: Local: All broker connections are down: 1/1 brokers are down : terminating

我让 Kafka 监听端口 9092：

bash-3.2$ netstat -an | grep 9092
tcp4       0      0  127.0.0.1.9092         127.0.0.1.50994        ESTABLISHED
tcp4       0      0  127.0.0.1.50994        127.0.0.1.9092         ESTABLISHED
tcp4       0      0  127.0.0.1.9092         127.0.0.1.50986        ESTABLISHED
tcp4       0      0  127.0.0.1.50986        127.0.0.1.9092 …

我可以用来kafkacat从主题中获取元组并将其打印出来：

kafkacat -b kafka10.myorg.com:9092 -t MyTopic -o -1 -f '%s\\n'\n

我在终端中得到类似这样的信息：

\xef\xbf\xbd\xc7\x90\xdb\x92\xef\xbf\xbd\xd2\x93\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xc7\x90\xdb\x92\xef\xbf\xbd\xd2\x93\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbdS1_153314S3_153314S4_5422973S2_5420991...\n

我是否可以将打印出的有效负载转换为二进制格式，以便我可以将其提供给本地代码进行调试？