hadoop map减少永远完成

Question

我是地图减少的世界的新手,我已经完成了一项工作,似乎需要永远完成,因为这是一项相对较小的任务,我猜测有些事情没有按计划进行.我正在使用hadoop 2.6版,这里收集了一些我认为可能有用的信息.地图简化程序本身很简单所以我不打算在这里添加它们,除非有人真的希望我提供更多的见解 - 运行map reduce的python代码与这里的相同 - http://www.michael-noll. com/tutorials/writing-an-hadoop-mapreduce-program-in-python /.如果有人能够提供出错的线索,或者为什么会出现问题.提前致谢.

Name:   streamjob1669011192523346656.jar
Application Type:   MAPREDUCE
Application Tags:   
State:  ACCEPTED
FinalStatus:    UNDEFINED
Started:    3-Jul-2015 00:17:10
Elapsed:    20mins, 57sec
Tracking URL:   UNASSIGNED
Diagnostics:    

这是我在运行程序时得到的:

bin/hadoop jar share/hadoop/tools/lib/hadoop-streaming-2.6.0.jar -  file python-files/mapper.py -mapper python-files/mapper.py -file python -    files/reducer.py -reducer python-files/reducer.py -input /user/input/* -  output /user/output
15/07/03 00:16:41 WARN streaming.StreamJob: -file option is deprecated, please use generic option -files instead.
2015-07-03 00:16:43.510 java[3708:1b03] Unable to load realm info from   SCDynamicStore
15/07/03 00:16:44 WARN util.NativeCodeLoader: Unable to load native-   hadoop library for your platform... using builtin-java classes where     applicable
packageJobJar: [python-files/mapper.py, python-files/reducer.py,     /var/folders/4x/v16lrvy91ld4t8rqvnzbr83m0000gn/T/hadoop-unjar8212926403009053963/] []     /var/folders/4x/v16lrvy91ld4t8rqvnzbr83m0000gn/T/streamjob1669011192523346656.jar tmpDir=null
15/07/03 00:16:53 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/07/03 00:16:55 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/07/03 00:17:05 INFO mapred.FileInputFormat: Total input paths to    process : 1
15/07/03 00:17:06 INFO mapreduce.JobSubmitter: number of splits:2
15/07/03 00:17:07 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1435852353333_0003
15/07/03 00:17:11 INFO impl.YarnClientImpl: Submitted application application_1435852353333_0003
15/07/03 00:17:11 INFO mapreduce.Job: The url to track the job:     http://mymacbook.home:8088/proxy/application_1435852353333_0003/
15/07/03 00:17:11 INFO mapreduce.Job: Running job: job_1435852353333_0003

Answer 1

ACCEPTED如果某个作业长时间处于状态且不改变RUNNING状态，可能是由于以下原因。

Nodemanager（从属服务）已死亡或无法与资源管理器通信。如果Active nodesYarn 资源管理器Web ui 主页中的 为零，那么您可以确认没有节点管理器连接到资源管理器。如果是这样，您需要启动nodemanager。

另一个原因是可能有其他作业正在运行，占用了可用槽位，没有空间容纳新作业，请检查资源管理器 webui 主页中Memory Total、Memory used、Vcores Total、的值。VCores Used