Chr*_*row 7 java maven google-cloud-platform google-cloud-dataflow apache-beam
(我也为此提出了一个GitHub问题 - https://github.com/googleapis/google-cloud-java/issues/4095)
我有Apache Beam的以下2个依赖项的最新版本:
依赖性1 - google-cloud-dataflow-java-sdk-all
(Apache Beam的分布旨在简化Google Cloud Dataflow服务上Apache Beam的使用 - https://mvnrepository.com/artifact/com.google.cloud.dataflow/google-cloud-dataflow-java-sdk-all)
<dependency>
<groupId>com.google.cloud.dataflow</groupId>
<artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
<version>2.5.0</version>
</dependency>
依赖关系2 - beam-runners-google-cloud-dataflow-java
(我猜这可以在Google Cloud Dataflow中实际运行Beam管道)
https://mvnrepository.com/artifact/org.apache.beam/beam-runners-google-cloud-dataflow-java
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>2.8.0</version>
</dependency>
Maven安装适用于这两个依赖项.我需要添加以下(第三)依赖项,以便独立订阅Apache Beam管道之外的Google Cloud pub/sub:
谷歌云,发布订阅
https://mvnrepository.com/artifact/com.google.cloud/google-cloud-pubsub
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-pubsub</artifactId>
<version>1.53.0</version>
</dependency>
(这是最新版本).添加完成后,我做了以下冲突mvn clean install:
Could not resolve version conflict among [com.google.cloud:google-cloud-pubsub:jar:1.53.0 -> com.google.cloud:google-cloud-core-grpc:jar:1.53.0 -> io.grpc:grpc-protobuf:jar:1.16.1 -> io.grpc:grpc-core:jar:1.16.1, com.google.cloud:google-cloud-pubsub:jar:1.53.0 -> com.google.cloud:google-cloud-core-grpc:jar:1.53.0 -> io.grpc:grpc-protobuf:jar:1.16.1 -> io.grpc:grpc-protobuf-lite:jar:1.16.1 -> io.grpc:grpc-core:jar:1.16.1, com.google.cloud:google-cloud-pubsub:jar:1.53.0 -> io.grpc:grpc-netty-shaded:jar:1.16.1 -> io.grpc:grpc-core:jar:[1.16.1,1.16.1], com.google.cloud:google-cloud-pubsub:jar:1.53.0 -> io.grpc:grpc-stub:jar:1.16.1 -> io.grpc:grpc-core:jar:1.16.1, com.google.cloud:google-cloud-pubsub:jar:1.53.0 -> io.grpc:grpc-auth:jar:1.16.1 -> io.grpc:grpc-core:jar:[1.16.1,1.16.1], com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.5.0 -> org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.5.0 -> io.grpc:grpc-core:jar:1.2.0, com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.5.0 -> org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.5.0 -> io.grpc:grpc-netty:jar:1.2.0 -> io.grpc:grpc-core:jar:[1.2.0,1.2.0], com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.5.0 -> org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.5.0 -> com.google.cloud.bigtable:bigtable-protos:jar:1.0.0-pre3 -> io.grpc:grpc-core:jar:1.5.0, com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.5.0 -> org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.5.0 -> com.google.cloud.bigtable:bigtable-client-core:jar:1.0.0 -> io.grpc:grpc-core:jar:1.7.0, com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.5.0 -> org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.5.0 -> com.google.cloud.bigtable:bigtable-client-core:jar:1.0.0 -> io.opencensus:opencensus-contrib-grpc-util:jar:0.7.0 -> io.grpc:grpc-core:jar:1.6.1, com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.5.0 -> org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.5.0 -> io.grpc:grpc-all:jar:1.2.0 -> io.grpc:grpc-core:jar:[1.2.0,1.2.0], com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.5.0 -> org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.5.0 -> io.grpc:grpc-all:jar:1.2.0 -> io.grpc:grpc-okhttp:jar:1.2.0 -> io.grpc:grpc-core:jar:[1.2.0,1.2.0], com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.5.0 -> org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.5.0 -> io.grpc:grpc-all:jar:1.2.0 -> io.grpc:grpc-protobuf-nano:jar:1.2.0 -> io.grpc:grpc-core:jar:1.2.0
如果我删除Apache Beam依赖项并保持依赖性,我不会得到任何Maven安装冲突google-cloud-pubsub.
有一个单独的谷歌"BOM"(物料清单)依赖管理器应该管理谷歌云依赖的子依赖,但这并没有解决我的冲突:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bom</artifactId>
<version>0.71.0-alpha</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
我按照这个问题的答案 -
我发现了这一点
com.google.cloud:google-cloud-pubsub:jar:1.53.0有一个子依赖:
io.grpc:grpc-core:jar:1.16.1
com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.5.0对各种版本的子依赖:
io.grpc:grpc-core:jar
(GRPC是一个高性能,开源的通用RPC框架)
如果我io.grpc:grpc-core:jar:1.6.1作为显式依赖项添加到我的POM文件中,它不会"固定"依赖项,因为它仍然作为子依赖项被拉入com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.5.0.
在下面的GitHub问题上,garrettjonesgoogle在2月21日确认google-cloud-bom没有固定io.grpc的版本:grpc-core:
https://github.com/googleapis/google-cloud-java/issues/2890
似乎Apache Beam依赖项管理io.grpc:grpc-core:jar内部的不同版本,但这与Google发布/订阅依赖项使用的(一致)版本冲突.
有什么办法可以解决这个问题吗？我可以使用Maven以某种方式隔离子依赖项吗？
我能够解决此问题，只需从Google Cloud-pubsub工件中排除grpc-core即可。此后，它以与Beam-runners-google-cloud-dataflow-java的依赖项相同的版本1.13.1并入依赖项树中。
这是对我有用的pom.xml的摘录。
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-pubsub</artifactId>
<version>1.53.0</version>
<exclusions>
<exclusion>
<groupId>io.grpc</groupId>
<artifactId>grpc-core</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>2.8.0</version>
</dependency>
</dependencies>
这并不是真正的解决方案，但我确实发现问题主要在于这种依赖性：
<dependency>
<groupId>com.google.cloud.dataflow</groupId>
<artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
<version>2.5.0</version>
</dependency>
该工件与 Apache Beam 和 Google pub sub 工件之间存在许多不一致的依赖关系io.grpc:grpc-core:jar和隔离，无法真正正常工作。google-cloud-pubsub无论使用什么版本的，其自身的依赖io.grpc:grpc-core:jar都会与 的版本之一发生冲突io.grpc:grpc-core:jar。
我针对 Google 工件提出了以下 Git 问题：
https://github.com/googleapis/google-cloud-java/issues/4095
我针对 Apache Beam 工件提出了以下几点（它指向我提出的 Google 问题）：
https://issues.apache.org/jira/browse/BEAM-6118
我正在通过解决这个问题来解决它。我可以轮询所需的数据，而不是从发布/订阅的订阅中使用数据，但这并不是最佳选择，并且会导致 Google Cloud 成本显着增加。