相关疑难解决方法(0)

import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
import play.api.libs.json._
import java.util.Date
import javax.xml.bind.DatatypeConverter
object Test {
def main(args:Array[String]): Unit = {
    val logFile="test.txt"
    val conf=new SparkConf().setAppName("Json Test")
    val sc = new SparkContext(conf)
    try {
        val out= "output/test"
        val logData=sc.textFile(logFile,2).map(line => Json.parse(cleanTypo(line))).cache()

    } finally { 
        sc.stop()
    }
}

由于有关Spark jackson冲突问题的说法,我使用mvn版本重建了Spark:use-latest-versions -Dincludes = org.codehaus.jackson:jackson-core-asl mvn versions:use-latest-versions -Dincludes = org .codehaus.jackson:杰克逊映射器,ASL

所以罐子已经更新到1.9.x但是我仍然有错误

15/03/02 03:12:19 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
java.lang.NoClassDefFoundError: org/codehaus/jackson/annotate/JsonClass
at      org.codehaus.jackson.map.introspect.JacksonAnnotationIntrospector.findDeserializationType(JacksonAnnotationIntrospector.java:524)
at org.codehaus.jackson.map.deser.BasicDeserializerFactory.modifyTypeByAnnotation(BasicDeserializerFactory.java:732)
at org.codehaus.jackson.map.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:427)
at org.codehaus.jackson.map.deser.StdDeserializerProvider._createDeserializer(StdDeserializerProvider.java:398)
at …