我正在提交具有以下规范的 Spark 作业：（已使用相同的程序运行从 50GB 到 400GB 的不同大小的数据范围）
/usr/hdp/2.6.0.3-8/spark2/bin/spark-submit
--master yarn
--deploy-mode cluster
--driver-memory 5G
--executor-memory 10G
--num-executors 60
--conf spark.yarn.executor.memoryOverhead=4096
--conf spark.shuffle.registration.timeout==1500
--executor-cores 3
--class classname /home//target/scala-2.11/test_2.11-0.13.5.jar
我在阅读时尝试过修复数据，并在通过 RDD 上的 Key 操作进行任何计数之前应用了修复：
val rdd1 = rdd.map(x=>(x._2._2,x._2._1)).distinct.repartition(300)
val receiver_count=rdd1.map(x=>x._2).distinct.count
用户类抛出异常：
org.apache.spark.SparkException: Job aborted due to stage failure: ShuffleMapStage 20 (repartition at data_prep.scala:87) has failed the maximum allowable number of times: 4. Most recent failure reason: org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output location for shuffle 9