我们已经在 mode 下创建了一个 3 节点的 kafka-3.3.1 集群kraft。这是基于bitnami-kafka图像。所有节点的基本配置为(每个节点的端口号不同,其他根据需要进行更改)
KAFKA_ENABLE_KRAFT: 'yes'
KAFKA_KRAFT_CLUSTER_ID: xxyyddjjjddkk1234
KAFKA_CFG_PROCESS_ROLES: broker,controller
KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_CFG_LISTENERS: CONTROLLER://:9093,INSIDE://:9092,EXTERNAL://:9094
KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,INSIDE:PLAINTEXT,EXTERNAL:PLAINTEXT
KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@dpkafka01:9093,2@dpkafka02:9093,3@dpkafka03:9093
KAFKA_CFG_ADVERTISED_LISTENERS: INSIDE://dpkafka02:9092,EXTERNAL://_{HOSTIP}:9098
KAFKA_BROKER_ID: 2
KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
KAFKA_HEAP_OPTS: "-Xmx1G -Xms256m"
KAFKA_LOG_DIRS: /bitnami/kafka/kafka-logs
KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'
KAFKA_LOG_RETENTION_MS: 7200000
KAFKA_LOG_SEGMENT_MS: 86400000
KAFKA_LOG_DELETE_RETENTION_MS: 7200000
KAFKA_LOG_RETENTION_CHECK_INTERVAL_MS: 60000
KAFKA_LOG_CLEANUP_POLICY: "compact,delete"
KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS: 12000
KAFKA_CFG_NUM_RECOVERY_THREADS_PER_DATA_DIR: 4
KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR: 2
KAFKA_CFG_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 2
KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR: 2
ALLOW_PLAINTEXT_LISTENER: 'yes'
BITNAMI_DEBUG: 'true'
KAFKA_OPTS: -javaagent:/opt/bitnami/kafka/libs/jmx_prometheus_javaagent.jar=7072:/opt/bitnami/kafka/libs/prom-jmx-agent-config.yml
Run Code Online (Sandbox Code Playgroud)
当集群工作一段时间后,其中一两个会频繁关闭。日志对于确定根本原因并不是很有帮助。在状态更改为关闭之前我们看到的一些相关日志是:
[2022-12-04 08:35:16,928] INFO [RaftManager nodeId=2] Become candidate due to fetch timeout (org.apache.kafka.raft.KafkaRaftClient)
[2022-12-04 08:35:17,414] INFO …Run Code Online (Sandbox Code Playgroud)