YARN Resourcemanager没有连接到nodemanager

Question

提前感谢您的帮助

我正在运行以下版本:

Hadoop 2.2 zookeeper 3.4.5 Hbase 0.96 Hive 0.12

当我转到http://:50070时,我能够正确地看到2个节点正在运行.

问题是,当我转到http://:8088时,它显示0个节点正在运行.

据我所知:8088反映了资源管理器并显示了运行的nodemanagers的数量.守护进程全部启动,但似乎节点管理器没有连接到资源管理器.

这是日志文件:

2013-12-16 20:55:48,648 INFO org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /127.0.0.1:8031
2013-12-16 20:55:49,755 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2013-12-16 20:55:50,756 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2013-12-16 20:55:51,757 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2013-12-16 20:55:52,758 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2013-12-16 20:55:53,759 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2013-12-16 20:55:54,760 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

我已检查并且端口8031已打开.

编辑:

对于将来查看此内容的人,我需要编辑我的yarn-site.xml,如下所示:

<property>
   <name>yarn.nodemanager.aux-services</name>
   <value>mapreduce_shuffle</value>
</property>
<property>
   <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
   <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
   <name>yarn.resourcemanager.scheduler.address</name>
   <value>master-1:8030</value>
</property>
<property>
   <name>yarn.resourcemanager.address</name>
   <value>master-1:8032</value>
</property>
<property>
   <name>yarn.resourcemanager.webapp.address</name>
   <value>master-1:8088</value>
</property>
<property>
   <name>yarn.resourcemanager.resource-tracker.address</name>
   <value>master-1:8031</value>
</property>
<property>
   <name>yarn.resourcemanager.admin.address</name>
   <value>master-1:8033</value>
</property> 

Answer 1

你需要指定

主机名:端口

是的,它必须在所有节点上设置,而不仅仅是ResourceManager节点(你可以将相同的hadoop的conf dir复制到所有节点).如果你需要一些最小的工作配置来开始,请看看这里:toster.ru/q/57046#answer_208326

Answer 2

我有一个非常类似的问题,它只是通过指定ResourceManager 主机名解决,不需要拼出每个服务的确切地址.

<property>
  <name>yarn.resourcemanager.hostname</name>
  <value>master-1</value>
</property>