Kafka 连接:提供了配置 XXX 但不是 AdminClientConfig 中的已知配置

Hun*_*orn 17 apache-kafka apache-kafka-connect

在启动 Kafka-Connect 时,我看到了很多警告

10:33:56.706 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'config.storage.topic' was supplied but isn't a known config.
10:33:56.707 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'group.id' was supplied but isn't a known config.
10:33:56.708 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'status.storage.topic' was supplied but isn't a known config.
10:33:56.709 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'internal.key.converter.schemas.enable' was supplied but isn't a known config.
10:33:56.710 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'config.storage.replication.factor' was supplied but isn't a known config.
10:33:56.710 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'offset.flush.interval.ms' was supplied but isn't a known config.
10:33:56.711 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'key.converter.schemas.enable' was supplied but isn't a known config.
10:33:56.712 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'internal.key.converter' was supplied but isn't a known config.
10:33:56.712 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'internal.value.converter.schemas.enable' was supplied but isn't a known config.
10:33:56.713 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'status.storage.replication.factor' was supplied but isn't a known config.
10:33:56.713 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'value.converter.schemas.enable' was supplied but isn't a known config.
10:33:56.714 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'internal.value.converter' was supplied but isn't a known config.
10:33:56.714 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'offset.storage.replication.factor' was supplied but isn't a known config.
10:33:56.715 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'offset.storage.topic' was supplied but isn't a known config.
10:33:56.715 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'value.converter' was supplied but isn't a known config.
10:33:56.716 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'key.converter' was supplied but isn't a known config.

Run Code Online (Sandbox Code Playgroud)

有些帖子说这是因为主题创建被禁用,但在我的情况下不是这样。存储主题仍然创建。

另一个提到它的版本不匹配,在我的情况下也不是。kafka broker 版本是 1.0.0,Kafka-Connect 和下面的 pom.xml 一样。这些配置是在 Kafka-Connect 启动期间提供的

<properties>
      <!-->
      Confluent version must match Kafka version
      https://docs.confluent.io/current/installation/versions-interoperability.html
      <-->
        <kafka.version>1.0.0</kafka.version>
        <confluent.version>5.1.2</confluent.version>
        <debezium-connector-mongodb.version>0.7.4</debezium-connector-mongodb.version>
        <logging.version>2.11.2</logging.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>connect-json</artifactId>
            <version>${kafka.version}</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>connect-runtime</artifactId>
            <version>${kafka.version}</version>
            <scope>runtime</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>io.debezium</groupId>
            <artifactId>debezium-connector-mongodb</artifactId>
            <version>${debezium-connector-mongodb.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>connect-file</artifactId>
            <version>${kafka.version}</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>io.confluent</groupId>
            <artifactId>kafka-connect-jdbc</artifactId>
            <version>${confluent.version}</version>
            <scope>runtime</scope>
            <exclusions>
                <exclusion>
                    <groupId>log4j</groupId>
                    <artifactId>log4j</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
Run Code Online (Sandbox Code Playgroud)

为了修复这些警告,我还应该检查什么?我可以看到数据是通过 Kafka-Connect 传送的,但仍然担心这些警告

Rob*_*att 12

这些只是WARNings,可以忽略。在未来的版本中,它们将被正确抑制:https : //github.com/apache/kafka/pull/5876

  • 我在 kafka 2.6.0 中仍然看到这些。忽略确实不是一个好的答案 - 日志应该是干净的,以便监控团队可以对有效的警告做出反应。 (8认同)
  • 在整个 Kafka 领域,跟踪用户提供的配置(以某种方式)可能会很有用,而不是完全抑制这些警告;如果用户故意提供配置,期望它会改变语义或处理,那么它可以很好地暗示您的期望可能不会实现:p ...可能是初衷 (2认同)