小编use*_*942的帖子

在pyspark中将行转换为Dictionary

我在pyspark中有一个DataFrame(df),通过从hive表中读取:

df=spark.sql('select * from <table_name>')


+++++++++++++++++++++++++++++++++++++++++++
|  Name    |    URL visited               |
+++++++++++++++++++++++++++++++++++++++++++
|  person1 | [google,msn,yahoo]           |
|  person2 | [fb.com,airbnb,wired.com]    |
|  person3 | [fb.com,google.com]          |
+++++++++++++++++++++++++++++++++++++++++++
Run Code Online (Sandbox Code Playgroud)

当我尝试以下,得到一个错误

df_dict = dict(zip(df['name'],df['url']))
"TypeError: zip argument #1 must support iteration."
Run Code Online (Sandbox Code Playgroud)

type(df.name) is of 'pyspark.sql.column.Column'

我如何创建如下的字典,以后可以迭代

{'person1':'google','msn','yahoo'}
{'person2':'fb.com','airbnb','wired.com'}
{'person3':'fb.com','google.com'}
Run Code Online (Sandbox Code Playgroud)

感谢您的想法和帮助.

pyspark

5
推荐指数
3
解决办法
9455
查看次数

标签 统计

pyspark ×1