我知道有一个非常相似的帖子(无法在hadoop二进制路径中找到winutils二进制文件),但是,我已经尝试了建议的每一步,但仍然出现相同的错误.
我正在尝试在Windows 7上使用Apache Spark版本1.6.0来执行此页面上的教程http://spark.apache.org/docs/latest/streaming-programming-guide.html,特别是使用此代码:
./bin/run-example streaming.JavaNetworkWordCount localhost 9999
Run Code Online (Sandbox Code Playgroud)
阅读本文后, 无法在hadoop二进制路径中找到winutils二进制文件
我意识到我需要winutils.exe文件,所以我用它下载了一个hadoop二进制2.6.0,定义了一个名为HADOOP_HOME的环境变量:
with value C:\Users\GERAL\Desktop\hadoop-2.6.0\bin
Run Code Online (Sandbox Code Playgroud)
并将其放在路径上,如下所示:%HADOOP_HOME%
但是当我尝试代码时仍会出现相同的错误.有谁知道如何解决这个问题?
问题与我之前的问题相关联所有守护进程都在运行,jps显示:
6663 JobHistoryServer
7213 ResourceManager
9235 Jps
6289 DataNode
6200 NameNode
7420 NodeManager
Run Code Online (Sandbox Code Playgroud)
但wordcount示例继续失败,出现以下异常:
ERROR security.UserGroupInformation: PriviledgedActionException as:root (auth:SIMPLE) cause:java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
Exception in thread "main" java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1238)
at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1234)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1233)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1262)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
at …Run Code Online (Sandbox Code Playgroud)