为什么IDEA找不到DS()和toDF()函数？

Question

我的代码在spark-shell中运行良好:

scala> case class Person(name:String,age:Int)
defined class Person

scala> val person = Seq(Person("ppopo",23)).toDS()
person: org.apache.spark.sql.Dataset[Person] = [name: string, age: int]

scala> person.show()
+-----+---+
| name|age|
+-----+---+
|ppopo| 23|
+-----+---+

但在IDEA错了:

我已经在"spark-2.0.0-bin-hadoop2.7/jars /"中导入了所有的罐子,但仍然找不到这个功能.

Answer 1

我发现问题,在使用toDS()之前添加一个依赖项:

val ss = SparkSession.builder().appName("DataSet Test")
  .master("local[*]").getOrCreate()

// This import is needed
import ss.implicits._
val simpleDS = Seq(Person("po",12)).toDS()
simpleDS.show()