java.lang.NoSuchFieldError: IBM_JAVA

Question

我看到很多对这个问题的引用，我已经在这里发布了

我正在使用 Hadoop 2.4.1 和 Flume 1.5.0.1。我的flume-env.sh配置如下

FLUME_CLASSPATH="/var/lib/apache-flume-ng:lib/hadoop-core-1.2.0.jar:lib/hadoop-auth-2.4.1.jar:lib/hadoop-yarn-api-2.4.1.jar:lib/hadoop-mapreduce-client-jobclient-2.4.1.jar:lib/hadoop-mapreduce-client-core-2.4.1.jar:lib/hadoop-common-2.4.1.jar:lib/hadoop-annotations-2.4.1.jar"

对于这些 jar，我又添加了一个 jar，它是在 Flume 的 lib 中可用的 commons-configuration-1.6.jar。我是 Flume 和 Hadoop 的新手。

完整的跟踪如下：

ERROR [conf-file-poller-0] (org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run:149)  - Unhandled error
java.lang.NoSuchFieldError: IBM_JAVA
    at org.apache.hadoop.security.UserGroupInformation.getOSLoginModuleName(UserGroupInformation.java:337)
    at org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:382)
    at org.apache.flume.sink.hdfs.HDFSEventSink.authenticate(HDFSEventSink.java:553)
    at org.apache.flume.sink.hdfs.HDFSEventSink.configure(HDFSEventSink.java:272)
    at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
    at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:418)
    at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:103)
    at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

Answer 1

该问题是由缺少或冲突的依赖项引起的：

1. 将 hadoop-auth 添加到您的类路径
2. 如果问题仍然存在，请从类路径中删除 hadoop-core。它与 hadoop-auth 相冲突。

这将解决问题。