eclipse中的Hadoop java.lang.NoClassDefFoundError

Question

我是hadoop的新手,只是从在线教程中设置我的hadoop环境.但是,当我尝试按照以下指南在eclipse中创建一个hadoop类时:http: //importantfish.com/how-to-run-hadoop-in-standalone-mode-using-eclipse-on-mac-os-x /

我可以运行hadoop示例罐,一切正常.但当我完成该博客中涉及的所有步骤并在eclipse中运行时,我得到NoClassDefFoundErr:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/util/Apps
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at org.apache.hadoop.mapred.LocalDistributedCacheManager.setup(LocalDistributedCacheManager.java:93)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:163)
    at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:731)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
    at WordCount.main(WordCount.java:82)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.util.Apps
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 24 more

另外,我检查了有关类似问题的其他帖子.有人提到你应该同时拥有common-logging-1.1.3.jar和common-logging-api-1.0.4.jar.我没有common-logging-api,只是找不到它,我试着尽可能地查看hadoop提供的每个文件夹.

你能告诉我如何解决这个问题吗？

谢谢!

Answer 1

你必须包括hadoop-yarn-common-xxxjar.在hadoop的lib目录中查找它.

问题是你使用的是更高版本的hadoop(2.x),这个版本使用的是纱线.

提示:纱线的jar文件实际上位于:

${HADOOP_HOME}/share/hadoop/yarn/*.jar