我们正在使用 Dataflow Flex 模板并遵循本指南 ( https://cloud.google.com/dataflow/docs/guides/templates/using-flex-templates ) 来暂存和启动作业。这在我们的环境中有效。但是,当我通过 SSH 连接到 Dataflow VM 并运行时,docker ps我看到它正在将不同的 docker 映像引用到我们在模板中指定的映像(以绿色下划线标出):

我正在启动的模板如下,工作是使用创建的gcloud beta dataflow flex-template run:
{
"image": "gcr.io/<MY PROJECT ID>/samples/dataflow/streaming-beam-sql:latest",
"metadata": {
"description": "An Apache Beam streaming pipeline that reads JSON encoded messages from Pub/Sub, uses Beam SQL to transform the message data, and writes the results to a BigQuery",
"name": "Streaming Beam SQL",
"parameters": [
{
"helpText": "Pub/Sub subscription to read from.",
"label": "Pub/Sub input subscription.",
"name": "inputSubscription",
"regexes": …Run Code Online (Sandbox Code Playgroud)