Sye*_*ram 3 pyspark pyspark-sql
def main(inputs, output):
sdf = spark.read.csv(inputs, schema=observation_schema)
sdf.registerTempTable('filtertable')
result = spark.sql("""
SELECT * FROM filtertable WHERE qflag IS NULL
""").show()
temp_max = spark.sql(""" SELECT date, station, value FROM filtertable WHERE (observation = 'TMAX')""").show()
temp_min = spark.sql(""" SELECT date, station, value FROM filtertable WHERE (observation = 'TMIN')""").show()
result = temp_max.join(temp_min, condition1).select(temp_max('date'), temp_max('station'), ((temp_max('TMAX')-temp_min('TMIN'))/10)).alias('Range'))
Run Code Online (Sandbox Code Playgroud)
错误:
Traceback (most recent call last):
File "/Users/syedikram/Documents/temp_range_sql.py", line 96, in <module>
main(inputs, output)
File "/Users/syedikram/Documents/temp_range_sql.py", line 52, in main
result = temp_max.join(temp_min, condition1).select(temp_max('date'), temp_max('station'), ((temp_max('TMAX')-temp_min('TMIN')/10)).alias('Range'))
AttributeError: 'NoneType' object has no attribute 'join'
Run Code Online (Sandbox Code Playgroud)
在联接操作上执行会给我带来Nonetype对象错误。联机查找无济于事,因为pyspark sql的在线文档很少。我在这里做错了什么?
| 归档时间: |
|
| 查看次数: |
2622 次 |
| 最近记录: |