未安装slf4j时启动预构建的spark-master时出错

Question

我正在使用ubuntu来设置独立的火花.但是当我使用预先构建的火花时,它未能找到slf4j包.

./spark-1.4.1-bin-without-hadoop/sbin/start-master.sh

Spark Command: /usr/lib/jvm/java-7-oracle//bin/java -cp /root/spark-1.4.1-bin-without-hadoop/sbin/../conf/:/root/spark-1.4.1-bin-without-hadoop/lib/spark-assembly-1.4.1-hadoop2.2.0.jar -Xms512m -Xmx512m -XX:MaxPermSize=256m org.apache.spark.deploy.master.Master --ip localhost --port 7077 --webui-port 8080
========================================
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/Logger
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2615)
        at java.lang.Class.getMethod0(Class.java:2856)
        at java.lang.Class.getMethod(Class.java:1668)
        at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        ... 6 more

对不起,我不应该使用spark-1.4.1-bin-without-hadoop,当我下载spark-1.4.1-bin-hadoop2.6时它会起作用.这可能是我使用的问题,而不是Spark.

Answer 1

一个简单的修复可以使用hadoop classpath命令中的类路径,如Spark文档中所建议的:

export SPARK_DIST_CLASSPATH=$(hadoop classpath)