我想出了一个例外:
错误yarn.ApplicationMaster:用户类抛出异常:org.apache.spark.SparkException:任务不可序列化org.apache.spark.SparkException:任务不可序列化在org.apache.spark.util.ClosureCleaner $ .ensureSerializable(ClosureCleaner.scala: 304)在org.apache.spark.util.ClosureCleaner $ .org $ apache $ spark $ util $ ClosureCleaner $$ clean(ClosureCleaner.scala:294)at org.apache.spark.util.ClosureCleaner $ .clean(ClosureCleaner.scala :122)org.apache.spark.SparkContext.clean(SparkContext.scala:2032)在org.apache的org.apache.spark.rdd.RDD $$ anonfun $ foreach $ 1.apply(RDD.scala:889).来自org.apache.spark.rdd.RDDOperationScope的org.apache.spark.rdd.RDDOperationScope $ .withScope(RDDOperationScope.scala:147)中的spark.rdd.RDD $$ anonfun $ foreach $ 1.apply(RDD.scala:888) $ .withScope(RDDOperationScope.scala:108)位于org.apache.spark.rdd.RDD.withScope(RDD.scala:306)org.apache.spark.rdd.RDD.foreach(RDD.scala:888)at com .boot $ .test(Boot.scala:60)at com.Boot $ .main(Boot.scala:36)at com.Boot.main(Boot.scala)在sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method .invoke(Method.java:606)at org.apache.spark.deploy.yarn.ApplicationMaster $$ anon $ 2.run(ApplicationMaster.scala:525)引起:java.io.NotSerializableException:org.apache.kafka.clients .producer.KafkaProducer序列化堆栈: - 对象不可序列化(类:org.apache.kafka.clients.producer.KafkaProducer,值:org.apache.kafka.clients.producer.KafkaProducer@77624599) - 字段(类:com.Boot) $$ anonfun $ test $ 1,name:producer $ 1,type:class org.apache.kafka.clients.producer.KafkaProducer) - object(class com.Boot $$ anonfun $ test $ 1,)org.apache.spark.serializer .serializationDebugger $ …