在测试期间,Pickle无法在django locmem缓存中存储对象？

Question

有点令我困惑的东西......

>>> from django.core.cache import get_cache
>>>
>>> cache = get_cache('django.core.cache.backends.locmem.LocMemCache')
>>>
>>> # Set the 'content' cache key to a string
>>> cache.set('content', 'a string')
>>> cache.get('content')
'a string'
>>>
>>> class TestObj(object):
...     pass
>>>
>>> a = TestObj()
>>> cache.set('content', a)
>>>
>>> # cache hasn't updated...
>>> cache.get('content')
'a string'
>>>
>>> cache.set('content', 1)
>>> # this is fine however..
>>> cache.get('content')
1
>>>

好的,因此缓存由于某种原因不接受对象.

# in locmem.py, set() method
try:
    pickled = pickle.dumps(new_value, pickle.HIGHEST_PROTOCOL)
    self._cache[key] = pickled
except pickle.PickleError:
    pass

这就是为什么,它显然击中了PickleError

>>> import pickle
>>> pickled = pickle.dumps(a, pickle.HIGHEST_PROTOCOL)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/lib/python2.7/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 396, in save_reduce
    save(cls)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 748, in save_global
    (obj, module, name))
PicklingError: Can't pickle <class 'TestObj'>: it's not found as __builtin__.TestObj

当然可以,但为什么会这样呢？它在python控制台中运行得很好,但不是django shell吗？

# Works fine in python shell...
>>> import pickle  
>>> class TestObj(object):
...     pass
... 
>>> testobj = TestObj()   
>>> pickled = pickle.dumps(testobj, pickle.HIGHEST_PROTOCOL)
>>> pickled
'\x80\x02c__main__\nTestObj\nq\x00)\x81q\x01}q\x02b.'
>>>

出现此问题是因为我正在尝试将Mock()对象存储在缓存中以进行测试.我不确定我是否会采取错误的方式......

Answer 1

问题是pickle通过引用序列化类,所以你能不能只使用一个更好的序列化器来通过序列化类定义而不是通过引用来腌制类？然后你会挑选一个模拟对象,然后选择类源代码,然后你就可以将它传递给django缓存了.我是作者dill,它是一个更好的序列化程序......还有作者klepto,它是一个缓存包......这正是我在SQL表,磁盘或内存中存储任何对象的方法缓存.

本质上(不是尝试这个,但猜测它是基于我自己的缓存包的经验),它应该像这样工作:

>>> from django.core.cache import get_cache
>>> import dill
>>>
>>> cache = get_cache('django.core.cache.backends.locmem.LocMemCache')
>>>
>>> # Set the 'content' cache key to a string
>>> cache.set('content', dill.dumps('a string'))
>>> dill.loads(cache.get('content'))
'a string'
>>>
>>> class TestObj(object):
...     pass
>>>
>>> a = TestObj()
>>> cache.set('content', dill.dumps(a))
>>>
>>> dill.loads(cache.get('content'))
<__main__.TestObj object at 0x10235e510>
>>>
>>> # this is pickling classes w/o using a reference
>>> dill.dumps(a)
'\x80\x02cdill.dill\n_create_type\nq\x00(cdill.dill\n_load_type\nq\x01U\x08TypeTypeq\x02\x85q\x03Rq\x04U\x07TestObjq\x05h\x01U\nObjectTypeq\x06\x85q\x07Rq\x08\x85q\t}q\n(U\r__slotnames__q\x0b]q\x0cU\n__module__q\rU\x08__main__q\x0eU\x07__doc__q\x0fNutq\x10Rq\x11)\x81q\x12}q\x13b.'
>>> # and here's using a reference, which is exactly how pickle does it
>>> dill.dumps(a, byref=True)
'\x80\x02c__main__\nTestObj\nq\x00)\x81q\x01}q\x02b.'

如果您想自己尝试一下,请在此处获取dill(和klepto):https://github.com/uqfoundation