如何判断哪个对象属性pickle失败？

Question

当你挑选一个具有一些无法腌制的属性的对象时,它将失败,并出现如下通用错误消息:

PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed

有没有办法告诉哪个属性导致异常？我使用的是Python 2.5.2.

即使我原则上理解问题的根本原因(例如在上面的例子中有一个实例方法),但仍然很难准确地指出它.在我的情况下,我已经定义了一个自定义__getstate__方法,但忘记了一个关键属性.这发生在嵌套对象的复杂结构中,因此我花了一些时间来识别坏属性.

根据要求,这里有一个简单的例子是pickle故意失败:

import cPickle as pickle
import new

class Test(object):
    pass

def test_func(self):
    pass

test = Test()
pickle.dumps(test)
print "now with instancemethod..."
test.test_meth = new.instancemethod(test_func, test)
pickle.dumps(test)

这是输出:

now with instancemethod...
Traceback (most recent call last):
  File "/home/wilbert/develop/workspace/Playground/src/misc/picklefail.py", line 15, in <module>
    pickle.dumps(test)
  File "/home/wilbert/lib/python2.5/copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects

不幸的是,没有提示该属性test_meth导致问题.

Answer 1

您可以针对Python提交错误,因为其中不包含更多有用的错误消息.在此期间,修改_reduce_ex()函数copy_reg.py.

if base is self.__class__:
    print self # new   
    raise TypeError, "can't pickle %s objects" % base.__name__

输出:

<bound method ?.test_func of <__main__.Test object at 0xb7f4230c>>
Traceback (most recent call last):
  File "nopickle.py", line 14, in ?
    pickle.dumps(test)
  File "/usr/lib/python2.4/copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects

Answer 2

我和你有同样的问题,但是我的类有点复杂(即一个类似对象的大树),所以打印没有多大帮助,所以我一起攻击了一个辅助函数.它不完整,仅用于酸洗方案2:它足以让我找到我的问题.如果你想扩展它以涵盖所有内容,协议在http://www.python.org/dev/peps/pep-0307/中描述我已经使这个帖子可编辑,所以每个人都可以更新代码.

import pickle
def get_pickling_errors(obj,seen=None):
    if seen == None:
        seen = []
    try:
        state = obj.__getstate__()
    except AttributeError:
        return
    if state == None:
        return
    if isinstance(state,tuple):
        if not isinstance(state[0],dict):
            state=state[1]
        else:
            state=state[0].update(state[1])
    result = {}    
    for i in state:
        try:
            pickle.dumps(state[i],protocol=2)
        except pickle.PicklingError:
            if not state[i] in seen:
                seen.append(state[i])
                result[i]=get_pickling_errors(state[i],seen)
    return result

一个例子,其中K是不腌制的对象

>>> get_pickling_errors(K)
{'_gen': {}, '_base': {'_gens': None}}

这意味着attibute K._gen不可选,K._base._gens也是如此.