小编Sop*_*hie的帖子

如何在分布式模式下部署kafka connect?

我正在使用 kubernetes 中的 JDBC sink 连接器构建 Kafka-connect 应用程序。我尝试了独立模式,它正在工作。我想转向分布式模式。我可以通过运行下面的 yaml 文件成功构建两个 pod(kafka 连接器):

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: vtq
  name: kafka-sink-postgres-dis
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: kafka-sink-postgres-dis
    spec:
      containers:
      - name: kafka-sink-postgres-dis
        image: ***
        imagePullPolicy: Always
Run Code Online (Sandbox Code Playgroud)

bin/connect-distributed.sh config/worker.properties

bootstrap.servers=***:9092
offset.flush.interval.ms=10000

rest.port=8083
rest.host.name=127.0.0.1


key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=http://schema-registry:8081


# Prevent the connector from pulling all historical messages
auto.offset.reset=latest

# options below may be required for distributed mode

# unique name for the cluster, used in forming the Connect cluster group. Note …
Run Code Online (Sandbox Code Playgroud)

distributed apache-kafka-connect

5
推荐指数
0
解决办法
2793
查看次数

标签 统计

apache-kafka-connect ×1

distributed ×1