创建RDD时找不到火花错误RDD类型

Question

我正在尝试创建案例类对象的RDD.例如.,

// sqlContext from the previous example is used in this example.
// createSchemaRDD is used to implicitly convert an RDD to a SchemaRDD.
import sqlContext.createSchemaRDD

val people: RDD[Person] = ... // An RDD of case class objects, from the previous example.

// The RDD is implicitly converted to a SchemaRDD by createSchemaRDD, allowing it to be stored using        Parquet.
people.saveAsParquetFile("people.parquet")

我试图通过给出来完成上一个例子中的部分

    case class Person(name: String, age: Int)

    // Create an RDD of Person objects and register it as a table.
    val people: RDD[Person] = sc.textFile("/user/root/people.txt").map(_.split(",")).map(p => Person(p(0), p(1).trim.toInt))
    people.registerAsTable("people")

我收到以下错误:

<console>:28: error: not found: type RDD
       val people: RDD[Person] =sc.textFile("/user/root/people.txt").map(_.split(",")).map(p => Person(p(0), p(1).trim.toInt))

什么出错了？提前致谢!

Answer 1

这里的问题是显式RDD[String]类型注释.看起来RDD默认情况下不会导入spark-shell,这就是Scala抱怨无法找到RDD类型的原因.先试试吧import org.apache.spark.rdd.RDD.