小编use*_*607的帖子

kafka __consumer_offsets 主题分区数过多

我正在使用 Kafka 0.8.2,我的消费者收到一条错误消息“偏移提交失败......”。当查看主题“__consumer_offsets”时。我看到它有 50 个分区。这正常吗?我只能通过删除所有 Kafka 日志并重新启动 Kafka 服务器来解决此问题。有没有办法在这个主题达到一定数量的分区时删除它,或者我是否提交了错误的偏移量?

以下是我提交偏移量的方式:

 public void commitOffsets(BlockingChannel channel, String topic, String    groupid, int partition, String clientName, int corrilationid, long offset)   throws Exception{

    if (commitTryCount > 100){
        throw new Exception("Offset commit failed with " + channel.host());
    }

    long now = System.currentTimeMillis();
    Map<TopicAndPartition, OffsetAndMetadata> offsets = new LinkedHashMap<TopicAndPartition, OffsetAndMetadata>();
    //for (int i = 0; i < this.totalPartitions; i++){
        TopicAndPartition topicPartition = new TopicAndPartition(topic, partition);
        offsets.put(topicPartition, new OffsetAndMetadata(offset, topic, now));
    //}     

    //initialize offset commit
    OffsetCommitRequest commitRequest = …
Run Code Online (Sandbox Code Playgroud)

apache-kafka kafka-consumer-api

5
推荐指数
1
解决办法
5409
查看次数

标签 统计

apache-kafka ×1

kafka-consumer-api ×1