如何在spark Dataframe中像SQL一样实现EXISTS条件

Question

我很想知道，如何以 Spark Dataframe 方式实现类似 SQL 的 Exists 子句。

Answer 1

LEFT SEMI JOIN相当于EXISTSSpark中的功能。

val cityDF= Seq(("Delhi","India"),("Kolkata","India"),("Mumbai","India"),("Nairobi","Kenya"),("Colombo","Srilanka")).toDF("City","Country")

val CodeDF= Seq(("011","Delhi"),("022","Mumbai"),("033","Kolkata"),("044","Chennai")).toDF("Code","City")

val finalDF= cityDF.join(CodeDF, cityDF("City") === CodeDF("City"), "left_semi")