从Windows机器提交时,Mapreduce作业失败

Question

我试图从Windows机器上提交一个M/R作业到linux上的hadoop集群.我正在使用hadoop 2.2.0(HDP 2.0).

我收到以下错误:

2014-06-06 08:32:37,684 [main] INFO  Job.monitorAndPrintJob  - Job job_1399458460502_0053 running in uber mode : false
2014-06-06 08:32:37,704 [main] INFO  Job.monitorAndPrintJob  -  map 0% reduce 0%
2014-06-06 08:32:37,717 [main] INFO  Job.monitorAndPrintJob  - Job job_1399458460502_0053 failed with state FAILED due to: Application application_1399458460502_0053 failed 2 times due to AM Container for appattempt_1399458460502_0053_000002 exited with  exitCode: 1 due to: Exception from container-launch: 
org.apache.hadoop.util.Shell$ExitCodeException: /bin/bash: line 0: fg: no job control

    at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
    at org.apache.hadoop.util.Shell.run(Shell.java:379)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)


.Failing this attempt.. Failing the application.

java.io.IOException: Job failed!

我使用从群集中获取的配置,因此应该完全匹配.

任何提示或线索,可能是什么错？

谢谢

Answer 1

conf.set("mapreduce.app-submission.cross-platform", "true");

Answer 2

我找到了解决方案。官方已报告bug

https://issues.apache.org/jira/browse/MAPREDUCE-4052

讨论中提到了解决方法。解决方法对我来说效果很好。似乎那些在客户端解决问题的错误一直都是绿色的。