相关疑难解决方法(0)

如何为独立群集非hdfs模式启用火花历史记录服务器

我在独立模式下按照http://paxcel.net/blog/how-to-setup-apache-spark-standalone-cluster-on-multiple-machine/设置了Spark2.1.1集群(1个主设备2个从设备)。我的机器上没有Hadoop之前的设置。我想启动火花历史记录服务器。我按如下方式运行:

roshan@bolt:~/spark/spark_home/sbin$ ./start-history-server.sh
Run Code Online (Sandbox Code Playgroud)

在spark-defaults.conf中,我将其设置为:

spark.eventLog.enabled           true
Run Code Online (Sandbox Code Playgroud)

但是它失败并显示以下错误:

7/06/29 22:59:03 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(roshan); groups with view permissions: Set(); users  with modify permissions: Set(roshan); groups with modify permissions: Set()
17/06/29 22:59:03 INFO FsHistoryProvider: History server ui acls disabled; users with admin permissions: ; groups with admin permissions
Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:278)
    at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala)
Caused by: java.io.FileNotFoundException: Log directory …
Run Code Online (Sandbox Code Playgroud)

apache-spark pyspark

2
推荐指数
1
解决办法
2026
查看次数

标签 统计

apache-spark ×1

pyspark ×1