小编Chi*_*asu的帖子

类中的 Pyspark User-Defined_functions

我正在尝试在 python 类中创建一个 Spark-UDF。意思是,类中的方法之一是 UDF。我收到一个名为“ PicklingError:无法序列化对象:TypeError:无法pickle _MovedItems对象”的错误

Environment : Azure Databricks . (DBR version 6.1 Beta) Code execution : In the built in Notebook. Python version : 3.5 Spark version : 2.4.4

I have tried defining the UDF outside of the class in a separate cell, and the UDF works. I do not want to write code like that, I need to follow OOP principles and would like to keep it structured. I have tried everything on Google, did not …

python-3.x pyspark jupyter-notebook azure-databricks

5
推荐指数
2
解决办法
3868
查看次数