下面是代码
def findUniqueGroupInMetadata(sc: SparkContext): Unit = {
val merchantGroup = sc.cassandraTable("local_pb", "merchant_metadata").select("group_name")
try {
val filterByWithGroup = merchantGroup.filter {
row =>
row.getStringOption("group_name") match {
case Some(s: String) if (s != null) => true
case None => false
}
}.map(row => row.getStringOption("group_name").get.capitalize)
//filterByWithGroup.take(15).foreach(data => println("merchantGroup => " + data))
filterByWithGroup.saveToCassandra("local_pb", "merchant_group", SomeColumns("group_name"))
} catch {
case e: Exception => println(e.printStackTrace())
}
Run Code Online (Sandbox Code Playgroud)
}
例外=>
java.lang.IllegalArgumentException: Multiple constructors with the same number of parameters not allowed.
at com.datastax.spark.connector.util.Reflect$.methodSymbol(Reflect.scala:16)
at com.datastax.spark.connector.util.ReflectionUtil$.constructorParams(ReflectionUtil.scala:63)
at com.datastax.spark.connector.mapper.DefaultColumnMapper.<init>(DefaultColumnMapper.scala:45)
at …Run Code Online (Sandbox Code Playgroud) 我试图在我的本地启动DSE 5.0.1 Cassandra(单节点).
得到以下错误:
CassandraDaemon.java:698 - 如果snitch的数据中心(Cassandra)与以前的数据中心(Graph)不同,则无法启动节点.请修复snitch配置,停用并重新启动此节点或使用标志-Dcassandra.ignore_dc = true
我有一个表模式
appname text,
randomnum int,
addedtime timestamp,
shortuuid text,
assetname text,
brandname text,
PRIMARY KEY ((appname, randomnum), addedtime, shortuuid)
Run Code Online (Sandbox Code Playgroud)
addedtime是集群密钥
现在,当我在集群密钥添加时使用下推过滤器时,我看不到它被应用了
val rdd = tabledf.filter("addedtime > '" + _to + "'").explain
== Physical Plan ==
Filter (cast(addedtime#2 as string) > 2016-12-20 11:00:00)
Run Code Online (Sandbox Code Playgroud)
它也在spark cassandra连接器1.4中工作,但没有使用最新的一个cassandra连接器1.6.0-M1.请让我知道这个问题