PicklingError:不能pickle <class'minit.Decimal'>:它与decimal.Decimal不是同一个对象

Question

这是我今天在http://filmaster.com"> filmaster.com上遇到的错误:

PicklingError:不能pickle:它与decimal.Decimal不是同一个对象

这究竟意味着什么？它似乎没有多大意义......它似乎与django缓存有关.你可以在这里看到整个追溯:

Traceback(最近一次调用最后一次):

文件"/home/filmaster/django-trunk/django/core/handlers/base.py",第92行,在get_response response = callback(request,*callback_args,**callback_kwargs)

在show_film
workflow.set_data_for_authenticated_user()中输入文件"/home/filmaster/film20/film20/core/film_views.py",第193行

文件"/home/filmaster/film20/film20/core/film_views.py",第518行,在set_data_for_authenticated_user
object_id = self.the_film.parent.id)

在get_others_ratings
set_cache 中输入文件"/home/filmaster/film20/film20/core/film_helper.py",第179行(CACHE_OTHERS_RATINGS,str(object_id)+"_"+ str(user_id),userratings)

文件"/home/filmaster/film20/film20/utils/cache_helper.py",第80行,在set_cache中返回cache.set(CACHE_MIDDLEWARE_KEY_PREFIX + full_path,result,get_time(cache_string))

文件"/home/filmaster/django-trunk/django/core/cache/backends/memcached.py",第37行,设置为
self._cache.set(smart_str(key),value,timeout或self.default_timeout)

文件"/usr/lib/python2.5/site-packages/cmemcache.py",第128行,在set val中,flags = self._convert(val)

文件"/usr/lib/python2.5/site-packages/cmemcache.py",第112行,在_convert val = pickle.dumps(val,2)

PicklingError:不能pickle:它与decimal.Decimal不是同一个对象

Filmaster的源代码可以从这里下载:bitbucket.org/filmaster/filmaster-test

任何帮助将不胜感激.

Answer 1

在jupyter笔记本中运行时出现此错误.我认为问题在于我正在使用%load_ext autoreload autoreload 2.重新启动我的内核并重新运行解决了这个问题.

Answer 2

Pickle的一个奇怪之处在于,在你挑选其中一个实例之前导入类的方式可以巧妙地改变pickle对象.Pickle要求你在腌制它之前和取消它之前相同地导入对象.

例如:

from a.b import c
C = c()
pickler.dump(C)

将制作一个微妙的不同对象(有时):

from a import b
C = b.c()
pickler.dump(C)

尝试摆弄你的导入,它可能会纠正这个问题.

Answer 3

我也无法解释为什么这会失败，但是我自己解决此问题的解决方案是更改我的所有代码

from point import Point

到

import point

这一改变就奏效了。我很想知道为什么... ...

Answer 4

您是否以某种方式reload(decimal)或对小数模块进行了猴子补丁以更改小数类？这是最有可能产生这种问题的两件事。

Answer 5

multiprocessing通过调用启动进程可能会出现问题__init__。这是一个演示：

import multiprocessing as mp

class SubProcClass:
    def __init__(self, pipe, startloop=False):
        self.pipe = pipe
        if startloop:
            self.do_loop()

    def do_loop(self):
        while True:
            req = self.pipe.recv()
            self.pipe.send(req * req)

class ProcessInitTest:
    def __init__(self, spawn=False):
        if spawn:
            mp.set_start_method('spawn')
        (self.msg_pipe_child, self.msg_pipe_parent) = mp.Pipe(duplex=True)

    def start_process(self):
        subproc = SubProcClass(self.msg_pipe_child)
        self.trig_proc = mp.Process(target=subproc.do_loop, args=())
        self.trig_proc.daemon = True
        self.trig_proc.start()

    def start_process_fail(self):
        self.trig_proc = mp.Process(target=SubProcClass.__init__, args=(self.msg_pipe_child,))
        self.trig_proc.daemon = True
        self.trig_proc.start()

    def do_square(self, num):
        # Note: this is an synchronous usage of mp,
        # which doesn't make sense. But this is just for demo
        self.msg_pipe_parent.send(num)
        msg = self.msg_pipe_parent.recv()
        print('{}^2 = {}'.format(num, msg))

现在，使用上面的代码，如果我们运行这个：

if __name__ == '__main__':
    t = ProcessInitTest(spawn=True)
    t.start_process_fail()
    for i in range(1000):
        t.do_square(i)

我们得到这个错误：

Traceback (most recent call last):
  File "start_class_process1.py", line 40, in <module>
    t.start_process_fail()
  File "start_class_process1.py", line 29, in start_process_fail
    self.trig_proc.start()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/context.py", line 212, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/context.py", line 274, in _Popen
    return Popen(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/popen_spawn_posix.py", line 33, in __init__
    super().__init__(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/popen_fork.py", line 21, in __init__
    self._launch(process_obj)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/popen_spawn_posix.py", line 48, in _launch
    reduction.dump(process_obj, fp)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/reduction.py", line 59, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function SubProcClass.__init__ at 0x10073e510>: it's not the same object as __main__.__init__

如果我们将其更改为使用fork而不是spawn：

if __name__ == '__main__':
    t = ProcessInitTest(spawn=False)
    t.start_process_fail()
    for i in range(1000):
        t.do_square(i)

我们得到这个错误：

Process Process-1:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
TypeError: __init__() missing 1 required positional argument: 'pipe'

但是如果我们调用start_process方法，它不会__init__在mp.Process目标中调用，像这样：

if __name__ == '__main__':
    t = ProcessInitTest(spawn=False)
    t.start_process()
    for i in range(1000):
        t.do_square(i)

它按预期工作（无论我们使用spawn还是fork）。

Answer 6

我将用Python2.7中的简单Python类演示问题：

In [13]: class A: pass  
In [14]: class B: pass

In [15]: A
Out[15]: <class __main__.A at 0x7f4089235738>

In [16]: B
Out[16]: <class __main__.B at 0x7f408939eb48>

In [17]: A.__name__ = "B"

In [18]: pickle.dumps(A)
---------------------------------------------------------------------------
PicklingError: Can't pickle <class __main__.B at 0x7f4089235738>: it's not the same object as __main__.B

显示此错误的原因是我们试图转储A，但是由于我们将其名称更改为引用另一个对象“ B”，因此，泡菜实际上与要转储的对象-类A或B混淆。显然，泡菜人非常聪明，他们已经对此行为进行了检查。

解决方案：检查您要转储的对象是否与另一个对象具有冲突的名称。

我已经在下面用ipython和ipdb演示了上述情况的调试：

PicklingError: Can't pickle <class __main__.B at 0x7f4089235738>: it's not the same object as __main__.B

In [19]: debug
> /<path to pickle dir>/pickle.py(789)save_global()
    787                 raise PicklingError(
    788                     "Can't pickle %r: it's not the same object as %s.%s" %
--> 789                     (obj, module, name))
    790
    791         if self.proto >= 2:

ipdb> pp (obj, module, name)               **<------------- you are trying to dump obj which is class A from the pickle.dumps(A) call.**
(<class __main__.B at 0x7f4089235738>, '__main__', 'B')
ipdb> getattr(sys.modules[module], name)   **<------------- this is the conflicting definition in the module (__main__ here) with same name ('B' here).**
<class __main__.B at 0x7f408939eb48>

我希望这可以节省一些麻烦！阿迪奥斯！