齐柏林飞艇的Hello世界失败了

Question

我刚刚安装了apache zeppelin(从git repo的最新源代码构建)并成功地看到它在端口10008中启动并运行.我用一行代码创建了一个新的笔记本

val a = "Hello World!"

并运行此段并看到以下错误

java.net.ConnectException:连接被拒绝在java.net.PlainSocketImpl.socketConnect(本机方法)在java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)在java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)在java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)在java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)在java.net.Socket.connect(Socket.java:589)在org.apache. thrift.transport.TSocket.open(TSocket.java:182)在org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)在org.apache.zeppelin.interpreter.remote.ClientFactory.create( ClientFactory.java:37)位于org.apache.commons.pool2.impl的org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60).GenericObjectPool.create(GenericObjectPool.java:861)在org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)在org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java: 363)在org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139)在org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:137)在org.apache.zeppelin位于org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java)的org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104)中的.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:257) :197)org.apache.zeppelin.scheduler.Job.run(Job.java:170)at org.apache.zeppelin.scheduler.RemoteScheduler $ JobRunner.运行(RemoteScheduler.java:304)在java.util.concurrent.Executors $ RunnableAdapter.call(Executors.java:511)在java.util.concurrent.FutureTask.run(FutureTask.java:266)在java.util.concurrent中.ScheduledThreadPoolExecutor $ ScheduledFutureTask.access $ 201(ScheduledThreadPoolExecutor.java:180)java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)ScheduledThreadPoolExecutor $ ScheduledFutureTask.access $ 201(ScheduledThreadPoolExecutor.java:180)java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745ScheduledThreadPoolExecutor $ ScheduledFutureTask.access $ 201(ScheduledThreadPoolExecutor.java:180)java.util.concurrent.ScheduledThreadPoolExecutor $ ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745

任何线索？

我的后端是火花1.5,我通过解释器的网络界面验证,齐柏林飞艇指向正确版本的火花并适应spark.home.

Answer 1

当Zeppelin尝试创建解释器时,错误可能也会导致错误.

Zeppelin在不同的进程中启动解释器并尝试连接到使用Thrift协议

在我的情况下,我尝试在spark-defaults.conf中为spark驱动程序分配5GB时出现此错误.在注释此行时(或分配4g或更少),它会被解析

#spark.driver.memory              5g

你可以看看这个JIRA ZEPPELIN-305

编辑:

此错误可能是由于阻止Spark解释器进程启动的任何原因造成的.最近,我在尝试将JMX选项添加到ZEPPELIN_JAVA_OPTS时得到了它,这导致解释器进程使用与Zeppelin进程相同的JMX端口.给"端口已经使用"错误

请检查Zeppelin日志(默认情况下,它们位于ZEPPELIN_DIR/logs /中,以查看Zeppelin尝试启动Spark Interpreter时发生的情况

Answer 2

我注意到指向 Spark 的 url 不正确。有一次，我纠正了它，效果很好。不管怎么说，还是要谢谢你。