我在spark sql中按行动进行分组.在某些行中包含具有不同ID的相同值.在这种情况下,我想选择第一行.
这是我的代码.
val highvalueresult = highvalue.select($"tagShortID", $"Timestamp", $"ListenerShortID", $"rootOrgID", $"subOrgID", $"RSSI_Weight_avg")
.groupBy("tagShortID", "Timestamp").agg(max($"RSSI_Weight_avg")
.alias("RSSI_Weight_avg"))
val t2 = averageDF.join(highvalueresult, Seq("tagShortID", "Timestamp", "RSSI_Weight_avg"))
Run Code Online (Sandbox Code Playgroud)
这是我的结果.
tag,timestamp,rssi,listner,rootorg,suborg
2,1496745906,0.7,3878,4,3
4,1496745907,0.6,362,4,3
4,1496745907,0.6,718,4,3
4,1496745907,0.6,1901,4,3
Run Code Online (Sandbox Code Playgroud)
在上面的结果为时间戳1496745907三个listner相同的rssi值.在这种情况下,我想选择第一行.