小编Sam*_*ien的帖子

我刚在Mac上安装了以下内容(Yosemite 10.10.3):

• oracle java 1.8 update 45
• 斯卡拉2.11.6
• spark 1.4(预编译版本:http://d3kbcqa49mib13.cloudfront.net/spark-1.4.0-bin-hadoop2.6.tgz)
• 源自zeppelin(https://github.com/apache/incubator-zeppelin)没有额外的配置,只是从模板中复制了创建的zeppelin-env.sh和zeppelin-site.xml.没有编辑.

我遵循安装指南:https://zeppelin.incubator.apache.org/docs/install/install.html

我已经没有问题地构建了zeppelin:

mvn clean install -DskipTests

开始了

./bin/zeppelin-daemon.sh start

打开http:// localhost:8080并打开Tutorial Notebook.以下是刷新代码段时发生的情况:

以下是webapp日志中md解释器的例外情况:

ERROR [2015-06-19 11:44:37,410] ({WebSocketWorker-8} NotebookServer.java[runParagraph]:534) - Exception from run
org.apache.zeppelin.interpreter.InterpreterException: **Interpreter md not found**
    at org.apache.zeppelin.notebook.Note.run(Note.java:269)
    at org.apache.zeppelin.socket.NotebookServer.runParagraph(NotebookServer.java:531)
    at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:119)
    at org.java_websocket.server.WebSocketServer.onWebsocketMessage(WebSocketServer.java:469)
    at org.java_websocket.WebSocketImpl.decodeFrames(WebSocketImpl.java:368)
    at org.java_websocket.WebSocketImpl.decode(WebSocketImpl.java:157)
    at org.java_websocket.server.WebSocketServer$WebSocketWorker.run(WebSocketServer.java:657)

重新启动解释器似乎不会导致错误.

我正在尝试使用Apache Zeppelin笔记本使用spark-csv [1]将CSV文件加载到Spark数据框中,并且在加载没有值的数字字段时,解析器将失败并且该行将被跳过.

我原本期望该行被加载,数据框中的值加载该行并将值设置为NULL,以便聚合只是忽略该值.

%dep
z.reset()
z.addRepo("my-nexus").url("<my_local_nexus_repo_that_is_a_proxy_of_public_repos>")
z.load("com.databricks:spark-csv_2.10:1.1.0")


%spark
import org.apache.spark.sql.SQLContext
import org.apache.spark.sql.types._
import com.databricks.spark.csv._
import org.apache.spark.sql.functions._

val schema = StructType(
    StructField("identifier", StringType, true) ::
    StructField("name", StringType, true) ::
    StructField("height", DoubleType, true) :: 
    Nil)

val sqlContext = new SQLContext(sc)
val df = sqlContext.read.format("com.databricks.spark.csv")
                        .schema(schema)
                        .option("header", "true")
                        .load("file:///home/spark_user/data.csv")

df.describe("height").show()

以下是数据文件的内容:/home/spark_user/data.csv

identifier,name,height
1,sam,184
2,cath,180
3,santa,     <-- note that there is not height recorded for Santa !

这是输出:

+-------+------+
|summary|height|
+-------+------+
|  count|     2|    <- 2 of 3 lines loaded, ie. sam and …