小编Ana*_*ale的帖子

我正在尝试使用命令从本地机器 CLI 在 minikube(Kubernetes) 上执行 spark-submit

spark-submit --master k8s://https://127.0.0.1:8001 --name cfe2 
--deploy-mode cluster --class com.yyy.Test --conf spark.executor.instances=2 --conf spark.kubernetes.container.image docker.io/anantpukale/spark_app:1.1 local://spark-0.0.1-SNAPSHOT.jar

我有一个基于 verison 2.3.0 的简单火花作业 jar。我还在 docker 和 minikube 中将它容器化并在虚拟机上运行。以下是异常堆栈：

Exception in thread "main" org.apache.spark.SparkException: Must specify the driver container image at org.apache.spark.deploy.k8s.submit.steps.BasicDriverConfigurationStep$$anonfun$3.apply(BasicDriverConfigurationStep.scala:51) at org.apache.spark.deploy.k8s.submit.steps.BasicDriverConfigurationStep$$anonfun$3.apply(BasicDriverConfigurationStep.scala:51) at scala.Option.getOrElse(Option.scala:121)  at org.apache.spark.deploy.k8s.submit.steps.BasicDriverConfigurationStep.<init>(BasicDriverConfigurationStep.scala:51)
        at org.apache.spark.deploy.k8s.submit.DriverConfigOrchestrator.getAllConfigurationSteps(DriverConfigOrchestrator.scala:82)
        at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication$$anonfun$run$5.apply(KubernetesClientApplication.scala:229)
        at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication$$anonfun$run$5.apply(KubernetesClientApplication.scala:227)
        at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2585)
        at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:227)
        at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:192)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:879)
      at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) 2018-04-06 13:33:52 INFO  ShutdownHookManager:54 - Shutdown hook called 2018-04-06 …

我在 Windows 7 之上安装了 virtual box 5.2。我按照 docker 快速入门指南安装了 docker。用于 Windows 的泊坞窗

由于我的机器位于公司代理后面，我设置

HTTP_PROXY=http://xxx:port
HTTPS_PROXY=https://xxx:port
NO_PROXT="192.168.99.1/24"

通过在创建 docker 机器时传递它，并在机器启动后进行导出。'docker-machine create -d virtualbox --engine-env HTTP_PROXY="http://xxx:80" --engine-env HTTPS_PROXY="https://xxx:80" --engine-env NO_PROXY="192.168.99.1/24" default

docker run hello-world显示以上错误。