我在一个独立的集群中运行Spark.与Master和2个工作节点在同一节点上的python驱动程序应用程序.业务逻辑是由在Worker节点上创建的执行程序运行的python代码.
如果其中一个遗嘱执行人死亡,我最终会陷入困境.如果我强行杀死Worker 0上的一个后端进程,Master输出:
16/06/07 16:20:35 ERROR TaskSchedulerImpl: Lost executor 1 on sparkslave0: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/07 16:20:35 WARN TaskSetManager: Lost task 2.0 in stage 0.0 (TID 2, sparkslave0): ExecutorLostFailure (executor 1 exited caused by one of the running tasks) Reason: Remote RPC client disassociated. Likely due to containers exceeding thresholds, or network issues. Check driver logs for WARN messages.
16/06/07 16:20:35 INFO DAGScheduler: Executor …