小编Bas*_*Bas的帖子

我下载了Spark 1.1.0,并使用"sbt assembly"构建它.我尝试运行示例Spark-Cassandra项目:

import org.apache.spark.{SparkContext, SparkConf}
import org.apache.spark.SparkContext._
import com.datastax.spark.connector._

object Test {
  def main (args: Array[String]) {
    val conf = new SparkConf(true)
      .set("spark.cassandra.connection.host", "127.0.0.1")

    val sc = new SparkContext("spark://127.0.0.1:7077", "test", conf)

    val rdd = sc.cassandraTable("test", "kv")
    println(rdd.count)
    println(rdd.first)
    println(rdd.map(_.getInt("value")).sum)
  }
}

通过执行sbin目录中的./start-master.sh来启动spark master.然后使用以下命令在同一台机器上启动从站:

./start-slave.sh 0 spark://127.0.0.1:7077

spark:// ..地址是从localhost:8080(spark仪表板)获取的地址.

这一切都很好,仪表板看到工人.然后我运行Scala程序,并获得ClassNotFoundException.这似乎有点误导,因为工作日志报告此错误:

14/10/29 12:23:05 ERROR EndpointWriter: AssociationError [akka.tcp://sparkWorker@localhost:33137] -> [akka.tcp://sparkExecutor@localhost:37279]: Error [Association failed with [akka.tcp://#
akka.remote.EndpointAssociationException: Association failed with [akka.tcp://sparkExecutor@localhost:37279]
Caused by: akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: Connection refused: localhost/127.0.0.1:37279
]

当我启动示例程序时,会出现以下警告:

14/10/29 12:22:31 …