我正在使用 Debezium (0.7.5) MySQL 连接器,如果我想使用选项更新此配置,我试图了解什么是最佳方法table.whitelist。
假设我创建了一个连接器,如下所示:
curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" http://debezium-host/connectors/ -d '
{
"name": "MyConnector",
"config": {
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"connect.timeout.ms": "60000",
"tasks.max": "1",
"database.hostname": "myhost",
"database.port": "3306",
"database.user": "***",
"database.password": "***",
"database.server.id": "3227197",
"database.server.name": "MyServer",
"database.whitelist": "myDb",
"table.whitelist": "myDb.table1,myDb.table2",
"database.history.kafka.bootstrap.servers": "kb0:9092,kb1:9092,kb2:9092",
"database.history.kafka.topic": "MyConnectorHistoryTopic",
"max.batch.size": "1024",
"snapshot.mode": "initial",
"decimal.handling.mode": "double"
}
}'
Run Code Online (Sandbox Code Playgroud)
一段时间后(2 周),我需要向myDb.table3此table.whitelist选项添加一个新表 ( ) (该表是旧表,它是在连接器之前创建的)
我尝试的是:
通过 API 更新命令:
curl -i -X …Run Code Online (Sandbox Code Playgroud) 我想我可能错过了一些配置,但我们正在尝试使用 Debezium 对一个包含大约 800 万条记录的表中的所有行进行快照,一段时间后它会停止。
连接器配置是:
{
"connector.class":"io.debezium.connector.mysql.MySqlConnector",
"database.user":"MyUser",
"database.server.id":"12345",
"tasks.max":"1",
"database.history.kafka.bootstrap.servers":"MyKafka:9092",
"database.history.kafka.topic":"MyConnectorHistory",
"database.server.name":"MyDbName",
"database.port":"3306",
"table.whitelist":"BigTable",
"decimal.handling.mode":"double",
"database.hostname":"***",
"database.password":"***",
"name":"MyConnector",
"database.whitelist":"MyDb",
"snapshot.mode":"initial_only",
"connect.timeout.ms":"60000"
}
Run Code Online (Sandbox Code Playgroud)
连接器开始扫描行:
April 24th 2019, 13:06:52.573 2019-04-24 16:06:52,569 INFO MySQL|MyDbName|snapshot Step 9: - 2040000 of 8609643 rows scanned from table 'MyDb.BigTable' after 00:59:29.129 [io.debezium.connector.mysql.SnapshotReader]
... other prints
April 24th 2019, 12:17:28.448 2019-04-24 15:17:28,447 INFO MySQL|MyDbName|snapshot Step 9: - 50000 of 8609643 rows scanned from table 'MyDb.BigTable' after 00:10:05.008 [io.debezium.connector.mysql.SnapshotReader]
April 24th 2019, 12:07:43.183 2019-04-24 15:07:43,183 INFO MySQL|MyDbName|snapshot …Run Code Online (Sandbox Code Playgroud)