小编cor*_*nds的帖子

Apache Spark 驱动程序日志未指定阶段取消的原因

我在 YARN 下的 AWS EMR 上运行 Apache Spark。该集群有 1 个 master 和 10 个 executor。经过几个小时的处理后,我的集群失败了,我去查看日志。

所以,我看到所有正在工作的执行者都试图同时终止任务(这是某人执行者的日志):

20/03/05 00:02:12 INFO Executor: Executor is trying to kill task 66.0 in stage 2.0 (TID 466), reason: Stage cancelled
20/03/05 00:02:12 INFO Executor: Executor is trying to kill task 65.0 in stage 2.0 (TID 465), reason: Stage cancelled
20/03/05 00:02:12 INFO Executor: Executor is trying to kill task 67.0 in stage 2.0 (TID 467), reason: Stage cancelled
20/03/05 00:02:12 INFO Executor: Executor is trying …
Run Code Online (Sandbox Code Playgroud)

amazon-emr hadoop-yarn apache-spark

7
推荐指数
0
解决办法
566
查看次数

标签 统计

amazon-emr ×1

apache-spark ×1

hadoop-yarn ×1