Kafka Producer 配置重试策略

Question

需要更改 Kafka Producer 配置的哪些参数，以便生产者应该： 1）重试 n 次 2）在同一消息的 n 间隔后，以防代理宕机。

我需要处理与此相关的情况：https : //github.com/rsyslog/rsyslog/issues/1052

Answer 1

您可以将“重试”设置为 n（次数）。但这还不够，您还需要研究其他配置，这些配置也可能因此受到影响或使其无效。

1）如果您的生产者的acks = 0则它将不起作用。因为 acks = 0 ？生产者不会等待任何类型的确认。在这种情况下，不能保证经纪人收到了记录。重试配置不会生效，因为无法知道是否发生了任何故障。

2）如果您正在寻找事件的有序交付，那么您需要将max.in.flight.requests.per.connection设置为 1。

3) retry.backoff.ms - 在尝试重试对给定主题分区的失败请求之前等待的时间。这避免了在某些故障情况下在紧密循环中重复发送请求。

4) request.timeout.ms - The configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.

5) delivery.timeout.ms - An upper bound on the time to report success or failure after a call to send() returns. This limits the total time that a record will be delayed prior to sending, the time to await acknowledgment from the broker (if expected), and the time allowed for retriable send failures. The producer may report failure to send a record earlier than this config if either an unrecoverable error is encountered, the retries have been exhausted, or the record is added to a batch which reached an earlier delivery expiration deadline.

Reference link: https://kafka.apache.org/documentation/#producerconfigs

Answer 2

假设您指的是 confluence-kafka-python 库，我相信您正在寻找的配置是：

1. 消息发送最大重试次数
2. 重试.退避.ms

请参阅此处的配置详细信息