我正在尝试运行由Spark结构化流组成的应用程序-来自kafka的数据输入。Spark版本是2.4.0，scala版本是2.12.7。我正在使用sbt制作多个胖子-我的项目是多模块项目。建立罐子不是问题。当我尝试spark-submit用我的罐子时，NoSuchMethodError发生了。
provided从删除了范围spark-sql-kafka-0-10。val sparkSqlKafka = "org.apache.spark" %% "spark-sql-kafka-0-10" % sparkVersion
assemblyMergeStrategy我在下面添加了一行。case "META-INF/services/org.apache.spark.sql.sources.DataSourceRegister" => MergeStrategy.concat
这是整个错误日志。
2019-01-08 11:55:12 ERROR ApplicationMaster:91 - User class threw exception: java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider org.apache.spark.sql.kafka010.KafkaSourceProvider could not be instantiated
java.util.ServiceConfigurationError: org.apache.spark.sql.sources.DataSourceRegister: Provider org.apache.spark.sql.kafka010.KafkaSourceProvider could not be instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:232)
at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:43)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:247)
at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
at scala.collection.AbstractTraversable.filter(Traversable.scala:104)
at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:630)
at org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:161) …
Run Code Online (Sandbox Code Playgroud)