Apache Livy不适用于本地jar文件

Question

我正在尝试使用spark-submit运行本地jar文件，效果很好。这是命令-

spark-submit --class "SimpleApp" --master local myProject/target/scala-2.11/simple-project_2.11-1.0.jar

但是当我尝试卷曲时

curl -X POST --data '{
 "file": "file:///home/user/myProject/target/scala-2.11/simple-project_2.11-1.0.jar",
 "className": "SimpleApp",
}'  
-H 
"Content-Type: application/json" 
http://server:8998/batches

它抛出错误

"requirement failed: Local path /home/user/myProject/target/scala-2.11/simple-project_2.11-1.0.jar cannot be added to user sessions."

这是我的livy.conf文件，因为有些文章建议更改一些内容。

# What host address to start the server on. By default, Livy will bind to all network interfaces.
livy.server.host = 0.0.0.0

# What port to start the server on.
livy.server.port = 8998

# What spark master Livy sessions should use.
livy.spark.master = local

# What spark deploy mode Livy sessions should use.
livy.spark.deploy-mode = client

# List of local directories from where files are allowed to be added to user sessions. By
# default it's empty, meaning users can only reference remote URIs when starting their
# sessions.
livy.file.local-dir-whitelist= /home/user/.livy-sessions/

这个你能帮我吗。

提前致谢。

Answer 1

我最近使用cURL创建错误请求时，从Apache Livy获得了本地文件读取的解决方案。我只是将“ file：//”中的文件读取协议替换为“ local：/”，因此对我有用。

curl -X POST --data '{
  "file": "local:/home/user/myProject/target/scala-2.11/simple-project_2.11-1.0.jar",
  "className": "SimpleApp",
}'  
-H 
"Content-Type: application/json" 
http://server:8998/batches

那是一个很小的错误，但是仍然无法从HDFS访问我的jar文件。

谢谢大家的帮助。