我正在尝试在 python 类中创建一个 Spark-UDF。意思是,类中的方法之一是 UDF。我收到一个名为“ PicklingError:无法序列化对象:TypeError:无法pickle _MovedItems对象”的错误
Environment : Azure Databricks . (DBR version 6.1 Beta) Code execution : In the built in Notebook. Python version : 3.5 Spark version : 2.4.4
I have tried defining the UDF outside of the class in a separate cell, and the UDF works. I do not want to write code like that, I need to follow OOP principles and would like to keep it structured. I have tried everything on Google, did not …