在 Spark-Submit 上获取多条消息“寻求最早并重置分区主题分区的偏移量”

Question

我正在虚拟机上运行spark 2.4.4并以批处理模式阅读kafka主题。什么样的kafka消费者配置会导致这种情况？

收到如下大量消息，并且 Spark 作业将永远运行。

    20/06/14 09:00:43 INFO SubscriptionState: [Consumer clientId=consumer-spark-kafka-relation-de56e437-85a0-4431-bb02-faca7b3487ab-executor-2, groupId=spark-kafka-relation-de56e437-85a0-4431-bb02-faca7b3487ab-executor] 
Seeking to EARLIEST offset of partition topic-12
    20/06/14 09:00:44 INFO SubscriptionState: [Consumer clientId=consumer-spark-kafka-relation-de56e437-85a0-4431-bb02-faca7b3487ab-executor-3, groupId=spark-kafka-relation-de56e437-85a0-4431-bb02-faca7b3487ab-executor] 
Resetting offset for partition topic-38 to offset 0.
    20/06/14 09:00:44 INFO SubscriptionState: [Consumer clientId=consumer-spark-kafka-relation-de56e437-85a0-4431-bb02-faca7b3487ab-executor-3, groupId=spark-kafka-relation-de56e437-85a0-4431-bb02-faca7b3487ab-executor] 
Seeking to LATEST offset of partition topic-38
    20/06/14 09:00:44 INFO SubscriptionState: [Consumer clientId=consumer-spark-kafka-relation-de56e437-85a0-4431-bb02-faca7b3487ab-executor-2, groupId=spark-kafka-relation-de56e437-85a0-4431-bb02-faca7b3487ab-executor] 
Resetting offset for partition topic-12 to offset 0.
    20/06/14 09:00:44 INFO SubscriptionState: [Consumer clientId=consumer-spark-kafka-relation-de56e437-85a0-4431-bb02-faca7b3487ab-executor-2, groupId=spark-kafka-relation-de56e437-85a0-4431-bb02-faca7b3487ab-executor] 
Seeking to LATEST offset of partition topic-12