小编Bru*_*nal的帖子

我想从功能追加到数据框“ df”上的新列get_distance：

def get_distance(x, y):
    dfDistPerc = hiveContext.sql("select column3 as column3, \
                                  from tab \
                                  where column1 = '" + x + "' \
                                  and column2 = " + y + " \
                                  limit 1")

    result = dfDistPerc.select("column3").take(1)
    return result

df = df.withColumn(
    "distance",
    lit(get_distance(df["column1"], df["column2"]))
)

但是，我得到这个：

TypeError: 'Column' object is not callable

我认为发生这种情况是因为x和y是Column对象，我需要转换为String在查询中使用。我对吗？如果是这样，我该怎么做？

我正在尝试在数据框上建立一个新列，如下所示：

l = [(2, 1), (1,1)]
df = spark.createDataFrame(l)

def calc_dif(x,y):
    if (x>y) and (x==1):
        return x-y

dfNew = df.withColumn("calc", calc_dif(df["_1"], df["_2"]))
dfNew.show()

但是，我得到：

Traceback (most recent call last):
  File "/tmp/zeppelin_pyspark-2807412651452069487.py", line 346, in <module>
Exception: Traceback (most recent call last):
  File "/tmp/zeppelin_pyspark-2807412651452069487.py", line 334, in <module>
  File "<stdin>", line 38, in <module>
  File "<stdin>", line 36, in calc_dif
  File "/usr/hdp/current/spark2-client/python/pyspark/sql/column.py", line 426, in __nonzero__
    raise ValueError("Cannot convert column into bool: please use '&' for 'and', '|' for 'or', " …