关闭tomcat服务器时无法加载org.apache.hadoop.util.ShutdownHookManager

Question

我有一个在tomcat上运行的简单Web应用程序.此Web应用程序将文件读​​写到HDFS.

我面临的问题是每次我使用停止服务器./bin/shutdown.sh.我无法加载hadoop shutdownHookManager异常.

我确定hadoop-common(包含ShutDownManager)在tomcat类路径中.

谁能帮我吗？

我得到的例外:

Oct 14, 2013 5:57:54 PM org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
SEVERE: The web application [] created a ThreadLocal with key of type [com.ebay.kernel.calwrapper.CalTransactionHelper$1] (value [com.ebay.kernel.calwrapper.CalTransactionHelper$1@3d5a1c5f]) and a value of type [com.ebay.kernel.calwrapper.CalTransactionHelper.Stack] (value [[]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol stop 
INFO: Stopping ProtocolHandler ["http-bio-8080"]
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol stop
INFO: Stopping ProtocolHandler ["ajp-bio-8009"]
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["http-bio-8080"]
Oct 14, 2013 5:57:54 PM org.apache.coyote.AbstractProtocol destroy
INFO: Destroying ProtocolHandler ["ajp-bio-8009"]
Oct 14, 2013 5:57:54 PM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already.  Could not load org.apache.hadoop.util.ShutdownHookManager$2.  The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600)
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
  at org.apache.hadoop.util.ShutdownHookManager.getShutdownHooksInOrder(ShutdownHookManager.java:124)
  at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:52)
Exception in thread "Thread-9" java.lang.NoClassDefFoundError: org/apache/hadoop/util/ShutdownHookManager$2
  at org.apache.hadoop.util.ShutdownHookManager.getShutdownHooksInOrder(ShutdownHookManager.java:124)
  at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:52)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.ShutdownHookManager$2
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1714)
  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
  ... 2 more

Answer 1

我在构建的 Maven Mojo 中也遇到同样的问题。我认为问题是由在 Web 应用程序中加载 hadoop 类的上下文类加载器引起的。WebAppClassLoader 类加载器看不到这些定义。当 JVM 关闭时，正在执行关闭钩子并调用 ShutdownHookManager，但它已加载到现已损坏的上下文类加载器中，这就是问题所在。

将 hadoop jar 放在 tomcat 的 lib 目录中可能会有所帮助。或者在启动 tomcat 时设置一些 *_CLASSPATH java 选项。这样，WebAppClassLoader 将具有您的类的定义。

否则，您可以在销毁上下文之前手动执行 hadoop 的关闭挂钩并忽略上述异常。