如何在Spark SQL中按降序排列？

Question

我尝试了df.orderBy("col1").show(10)但它按升序排序.df.sort("col1").show(10)也按降序排序.我查看了stackoverflow,我发现的答案都已过时或提交给RDD.我想在spark中使用原生数据帧.

Answer 1

您还可以通过导入spark sql函数对列进行排序

import org.apache.spark.sql.functions._
df.orderBy(asc("col1"))

要么

import org.apache.spark.sql.functions._
df.sort(desc("col1"))

导入sqlContext.implicits._

import sqlContext.implicits._
df.orderBy($"col1".desc)

要么

import sqlContext.implicits._
df.sort($"col1".desc)

Answer 2

它是在org.apache.spark.sql.DataFrame对sort的方法:

df.sort($"col1", $"col2".desc)

注意$并.desc在sort列内部对结果进行排序.

Answer 3

PySpark中最简单的方法是添加参数ascending = False:

df.orderBy("col1", ascending=False).show(10)

参考:http://spark.apache.org/docs/2.1.0/api/python/pyspark.sql.html#pyspark.sql.DataFrame.orderBy

Answer 4

import org.apache.spark.sql.functions.desc

df.orderBy(desc("columnname1"),desc("columnname2"),asc("columnname3"))

Answer 5

df.sort($"ColumnName".desc).show()