如何从命令行获取 Hadoop 作业的成功/失败状态？

Question

我正在使用带有 MRv1 的 CDH4。据我所知，没有用于检查已完成作业的“状态”的命令行工具。当我转到 Web 控制台作业详细信息页面时，我可以看到“状态：失败”或“状态：成功”。如果我运行mapred job -list all或mapred job -status job_201309231203_0011，既不指示“失败”或“成功”。

我错过了其他一些命令吗？

Answer 1

输出的前几行hadoop job -list all是：

X jobs submitted
States are:
        Running : 1     Succeded : 2    Failed : 3      Prep : 4
JobId   State   StartTime       UserName        Priority        SchedulingInfo

输出行如下所示：

job_201309171413_38136  1       1382455374980   somebody        NORMAL  0 running map tasks using 0 map slots. 0 additional slots reserved. 1 running reduce tasks using 1 reduce slots. 0 additional slots reserved.
job_201309171413_37222  2       1382430339635   somebody        NORMAL  0 running map tasks using 0 map slots. 0 additional slots reserved. 0 running reduce tasks using 0 reduce slots. 0 additional slots reserved.

第二列是State工作的。基于标题行，1手段Running和2手段Succeeded。这不是最清晰的格式：4 行标题，需要引用标题才能弄清楚状态代码的实际含义，并且无法仅获取一项工作的状态。

为特定作业解析此输出的最简单方法是：

$ job_id=job_201309171413_38136
$ hadoop job -list all | awk -v job_id=${job_id} 'BEGIN{OFS="\t"; FS="\t"; final_state="Unknown"} $0 == "States are:" {getline; for(i=1;i<=NF;i++) { split($i,s," "); states[s[3]] = s[1] }} $1==job_id { final_state=states[$2]; exit} END{print final_state}'
Running

$ job_id=job_201309171413_37222
$ hadoop job -list all | awk -v job_id=${job_id} 'BEGIN{OFS="\t"; FS="\t"; final_state="Unknown"} $0 == "States are:" {getline; for(i=1;i<=NF;i++) { split($i,s," "); states[s[3]] = s[1] }} $1==job_id { final_state=states[$2]; exit} END{print final_state}'
Succeeded

$ job_id=foobar
$ hadoop job -list all | awk -v job_id=${job_id} 'BEGIN{OFS="\t"; FS="\t"; final_state="Unknown"} $0 == "States are:" {getline; for(i=1;i<=NF;i++) { split($i,s," "); states[s[3]] = s[1] }} $1==job_id { final_state=states[$2]; exit} END{print final_state}'
Unknown