我在kubernetes集群上运行Spark 2.3作业
kubectl版本
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.3", GitCommit:"d2835416544f298c919e2ead3be3d0864b52323b", GitTreeState:"clean", BuildDate:"2018-02-09T21:51:06Z", GoVersion:"go1.9.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.3", GitCommit:"f0efb3cb883751c5ffdbe6d515f3cb4fbe7b7acd", GitTreeState:"clean", BuildDate:"2017-11-08T18:27:48Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
当我在k8s master上运行spark submit时,驱动程序pod停留在Waiting:PodInitializing状态.
如果我几乎平行地提交工作,即一个接一个地提交5个工作,就会发生这种情况.
我在运行驱动程序pod的节点上尝试了kubectl describe节点这就是我得到的,我确实看到资源有过度提交但是我希望kubernetes调度程序不安排节点中的资源是否过度提交或节点是否在未就绪状态,在这种情况下节点处于就绪状态但是如果节点处于未就绪状态,我会观察到相同的行为.
Name: **********
Roles: worker
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/hostname=****
node-role.kubernetes.io/worker=true
Annotations: node.alpha.kubernetes.io/ttl=0
volumes.kubernetes.io/controller-managed-attach-detach=true
Taints: <none>
CreationTimestamp: Tue, 31 Jul 2018 09:59:24 -0400
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
OutOfDisk False Tue, …