在我按下ctrl + C之前,spark-submit没有退出

Question

我运行这个spark命令,使用Hortonworks vm成功运行spark Scala程序.但是一旦作业完成,它就不会退出spark-submit命令,直到我按下ctrl + C. 为什么？

spark-submit --class SimpleApp --master yarn-client --num-executors 3 --driver-memory 512m --executor-memory12m --executor-cores 1 target/scala-2.10/application_2.10-1.0.jar /user/root/decks/largedeck.txt 

这是代码,我正在运行.

/* SimpleApp.scala */
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf

object SimpleApp {
  def main(args: Array[String]) {
    val logFile = "YOUR_SPARK_HOME/README.md" // Should be some file on your system
    val conf = new SparkConf().setAppName("Simple Application")
    val sc = new SparkContext(conf) 
    val cards = sc.textFile(args(0)).flatMap(_.split(" "))
    val cardCount = cards.count()
    println(cardCount)
  }
}

Answer 1

您必须在上下文中调用stop()以干净地退出程序.