NameNode地址的URI无效

Question

我试图建立一个ClouderaHadoop集群，与包含一个主节点namenode，secondarynamenode并且jobtracker，和含有另外两个节点datanode和tasktracker。该Cloudera版本是4.6，操作系统是Ubuntu的精确64。此外，该集群是从AWS实例创建的。ssh passwordless还已经Java安装了Oracle-7。

每当执行sudo service hadoop-hdfs-namenode start我都会得到：

2014-05-14 05:08:38,023 FATAL org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode join
java.lang.IllegalArgumentException: Invalid URI for NameNode address (check fs.defaultFS): file:/// has no authority.
        at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:329)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:317)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.getRpcServerAddress(NameNode.java:370)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser(NameNode.java:422)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:442)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:621)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:606)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1177)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1241)

我的core-site.xml：

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
   <property>
      <name>fs.defaultFS</name>
      <value>hdfs://<master-ip>:8020</value>
   </property>
</configuration>

mapred-site.xml：

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
   <property>
      <name>mapred.job.tracker</name>
      <value>hdfs://<master-ip>:8021</value>
   </property>
</configuration>

hdfs-site.xml：

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
   <property>
      <name>dfs.replication</name>
      <value>2</value>
   </property>
   <property>
      <name>dfs.permissions</name>
      <value>false</value>
   </property>
</configuration>

我尝试使用public ip，private-ip，public dns和fqdn，但结果是一样的。目录/etc/hadoop/conf.empty如下：

-rw-r--r-- 1 root root   2998 Feb 26 10:21 capacity-scheduler.xml
-rw-r--r-- 1 root hadoop 1335 Feb 26 10:21 configuration.xsl
-rw-r--r-- 1 root root    233 Feb 26 10:21 container-executor.cfg
-rwxr-xr-x 1 root root    287 May 14 05:09 core-site.xml
-rwxr-xr-x 1 root root   2445 May 14 05:09 hadoop-env.sh
-rw-r--r-- 1 root hadoop 1774 Feb 26 10:21 hadoop-metrics2.properties
-rw-r--r-- 1 root hadoop 2490 Feb 26 10:21 hadoop-metrics.properties
-rw-r--r-- 1 root hadoop 9196 Feb 26 10:21 hadoop-policy.xml
-rwxr-xr-x 1 root root    332 May 14 05:09 hdfs-site.xml
-rw-r--r-- 1 root hadoop 8735 Feb 26 10:21 log4j.properties
-rw-r--r-- 1 root root   4113 Feb 26 10:21 mapred-queues.xml.template
-rwxr-xr-x 1 root root    290 May 14 05:09 mapred-site.xml
-rw-r--r-- 1 root root    178 Feb 26 10:21 mapred-site.xml.template
-rwxr-xr-x 1 root root     12 May 14 05:09 masters
-rwxr-xr-x 1 root root     29 May 14 05:09 slaves
-rw-r--r-- 1 root hadoop 2316 Feb 26 10:21 ssl-client.xml.example
-rw-r--r-- 1 root hadoop 2251 Feb 26 10:21 ssl-server.xml.example
-rw-r--r-- 1 root root   2513 Feb 26 10:21 yarn-env.sh
-rw-r--r-- 1 root root   2262 Feb 26 10:21 yarn-site.xml

并slaves列出ip addresses两个从属计算机的：

<slave1-ip>
<slave2-ip>

执行中

update-alternatives --get-selections | grep hadoop
hadoop-conf                    auto     /etc/hadoop/conf.empty

我已经做了很多搜索，但没有得到任何可以帮助我解决问题的信息。有人可以提供任何线索吗？

Answer 1

我遇到了同样的问题，并通过格式化namenode进行了修复。下面是命令：

hdfs namenode -format

core-site.xml条目是：

<configuration>
   <property>
      <name>fs.defaultFS</name>
      <value>hdfs://localhost:9000</value>
   </property>
</configuration>

那肯定会解决问题。

Answer 2

确保您已按照链接中的指示正确设置 HADOOP_PREFIX 变量： http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html

即使我遇到了与您相同的问题，也通过设置此变量得到了纠正