为什么提交作业失败了"NoSuchMethodError:scala.runtime.VolatileObjectRef.zero()Lscala/runtime/VolatileObjectRef;"？

Question

我正在尝试提交一份火花工作

它以这种方式开始:

import javax.xml.parsers.{SAXParser, SAXParserFactory}

import org.apache.spark
import org.apache.spark.graphx.{Graph, Edge, VertexId}
import org.apache.spark.rdd.{PairRDDFunctions, RDD}
import org.apache.spark.storage.StorageLevel
import org.apache.spark.{SparkContext, SparkConf}
import scala.util.Try
import org.apache.log4j.{Level, Logger}


object MyApp {

  def main(args: Array[String]) {

    val sparkConf = new SparkConf().setAppName("MyApp")
    val sc = new SparkContext(sparkConf)

当我启动它时,我收到以下错误:

App > Exception in thread "main" java.lang.NoSuchMethodError: scala.runtime.VolatileObjectRef.zero()Lscala/runtime/VolatileObjectRef;
App > at MyApp$.main(MyApp.scala)
App > at MyApp.main(MyApp.scala)
App > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
App > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
App > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
App > at java.lang.reflect.Method.invoke(Method.java:606)
App > at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328)
App > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
App > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

我究竟做错了什么？

编辑:包括完整堆栈跟踪.使用Scala 2.10和Spark 1.2.0.奇怪的是,在我的罐子里,我有两节课.当我激发提交一个,它工作(这是一个4行虚拟工作),但当我运行较长的一个(约40行),如果失败,上面的错误

Answer 1

zero()对scala.runtime.VolatileObjectRef已在斯卡拉2.11被引入,你可能会有编译对斯卡拉2.11和运行在斯卡拉2.10运行时库.

看到

• v2.10:https://github.com/scala/scala/blob/2.10.x/src/library/scala/runtime/VolatileObjectRef.java
• v2.11:https://github.com/scala/scala/blob/2.11.x/src/library/scala/runtime/VolatileObjectRef.java