我是scala和spark的新手.今天我试着编写一些代码,让它在spark上运行,但是有一个例外.
此代码在本地scala中工作
import org.apache.commons.lang.time.StopWatch
import org.apache.spark.{SparkConf, SparkContext}
import scala.collection.mutable.ListBuffer
import scala.util.Random
def test(): List[Int] = {
val size = 100
val range = 100
var listBuffer = new ListBuffer[Int] // here throw an exception
val random = new Random()
for (i <- 1 to size)
listBuffer += random.nextInt(range)
listBuffer.foreach(x => println(x))
listBuffer.toList
}
但是当我把这段代码放入spark中时,会抛出一个异常说:
15/01/01 14:06:17 INFO SparkDeploySchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
Exception in thread "main" java.lang.NoSuchMethodError: scala.runtime.ObjectRef.create(Ljava/lang/Object;)Lscala/runtime/ObjectRef;
at com.tudou.sortedspark.Sort$.test(Sort.scala:35)
at com.tudou.sortedspark.Sort$.sort(Sort.scala:23)
at com.tudou.sortedspark.Sort$.main(Sort.scala:14) …我是Spark的新手，请尝试基于Spark和Spark Streaming编写一些示例代码。
到目前为止，我已经在spark中实现了排序功能，这是代码：
def sort(listSize: Int, slice: Int): Unit = {
val conf = new SparkConf().setAppName(getClass.getName)
val spark = new SparkContext(conf)
val data = genRandom(listSize)
val distData = spark.parallelize(data, slice)
val result = distData.sortBy(x => x, true)
val finalResult = result.collect()
val step5 = System.currentTimeMillis()
printlnArray(finalResult, 0, 10)
spark.stop()
}
/**
* generate random number
* @return
*/
def genRandom(listSize: Int): List[Int] = {
val range = 100000
var listBuffer = new ListBuffer[Int]
val random = new Random()
for (i …play 2.0模板引擎是否支持html页面中的简单计算.
让我们说,我创建了一个sum.scala.html页面:
@(a:String, b: String)
<html>
<head></head>
<body>
<h1> answer is getSum(@a,@b) </h1>
</body>
</html>
有什么方法可以通过某些功能"获得a和b的sum"吗？或者任何游戏2.0专家是否知道有关游戏2.0模板引擎计算的任何好主意？谢谢