我正在使用sbt程序集来创建一个可以在火花上运行的胖罐.有依赖性grpc-netty.spark上的Guava版本比所需的版本旧grpc-netty,我遇到了这个错误:java.lang.NoSuchMethodError:com.google.common.base.Preconditions.checkArgument.我能够通过在spark上将userClassPathFirst设置为true来解决此问题,但会导致其他错误.
如果我错了,请纠正我,但根据我的理解,如果我正确地进行着色,我不应该将userClassPathFirst设置为true.这是我现在的着色方式:
assemblyShadeRules in assembly := Seq(
ShadeRule.rename("com.google.guava.**" -> "my_conf.@1")
.inLibrary("com.google.guava" % "guava" % "20.0")
.inLibrary("io.grpc" % "grpc-netty" % "1.1.2")
)
libraryDependencies ++= Seq(
"org.scalaj" %% "scalaj-http" % "2.3.0",
"org.json4s" %% "json4s-native" % "3.2.11",
"org.json4s" %% "json4s-jackson" % "3.2.11",
"org.apache.spark" %% "spark-core" % "2.2.0" % "provided",
"org.apache.spark" % "spark-sql_2.11" % "2.2.0" % "provided",
"org.clapper" %% "argot" % "1.0.3",
"com.typesafe" % "config" % "1.3.1",
"com.databricks" %% "spark-csv" % "1.5.0",
"org.apache.spark" % "spark-mllib_2.11" % "2.2.0" % …