访问类的多处理代理的属性

Question

我有一个类,我想以只读的方式与池中的子进程共享,所以我准备了一个类的代理,但它没有用.以下是我的问题的简化示例.

from multiprocessing.managers import BaseManager

class TestClass:
    def __init__(self, a):
        self.a = a
    def b(self):
        print self.a

class MyManager(BaseManager): pass

MyManager.register('test', TestClass)

if __name__ == '__main__':
    manager = MyManager()
    manager.start()
    t = TestClass(1)
    print t.a
    mt = manager.test(2)
    mt.b()
    mt.a

当我运行此代码时,我得到:

1
2
Traceback (most recent call last):
  File "multiprocess_example_stackexchange.py", line 20, in <module>
    mt.a 
AttributeError: 'AutoProxy[test]' object has no attribute 'a'

似乎我无法通过代理直接访问共享对象的属性.是使用获取属性的方法的唯一方法,还是我做错了什么？

Answer 1

在Proxy由使用的对象multiprocessing.BaseManager和它的子类,通常只会暴露方法从他们指的是,没有属性的对象.现在,有multiprocessing.Manager().Namespace,它提供了一个Proxy子类,并提供访问属性,而不是方法.我们可以创建自己Proxy继承的类型,它允许访问我们的所有属性,以及访问我们的b函数:

from multiprocessing.managers import BaseManager, NamespaceProxy

class TestClass(object):
    def __init__(self, a):
        self.a = a

    def b(self):
        print self.a

class MyManager(BaseManager): pass

class TestProxy(NamespaceProxy):
    # We need to expose the same __dunder__ methods as NamespaceProxy,
    # in addition to the b method.
    _exposed_ = ('__getattribute__', '__setattr__', '__delattr__', 'b')

    def b(self):
        callmethod = object.__getattribute__(self, '_callmethod')
        return callmethod('b')

MyManager.register('test', TestClass, TestProxy)

if __name__ == '__main__':
    manager = MyManager()
    manager.start()
    t = TestClass(1)
    print t.a
    mt = manager.test(2)
    print mt.a
    mt.a = 5
    mt.b()

输出:

1
2
5

编辑:

如果您希望能够将原始类中的方法动态添加到Proxy类,则可以执行以下操作:

from multiprocessing.managers import BaseManager, NamespaceProxy
import inspect

class TestClass(object):
    def __init__(self, a):
        self.a = a

    def b(self):
        print self.a

class AnotherClass(object):
    def __init__(self, a):
        self.a = a

    def c(self):
        print self.a

class MyManager(BaseManager): pass

class ProxyBase(NamespaceProxy):
    _exposed_ = ('__getattribute__', '__setattr__', '__delattr__')

class TestProxy(ProxyBase): pass
class AnotherProxy(ProxyBase): pass


def register_proxy(name, cls, proxy):
    for attr in dir(cls):
        if inspect.ismethod(getattr(cls, attr)) and not attr.startswith("__"):
            proxy._exposed_ += (attr,)
            setattr(proxy, attr, 
                    lambda s: object.__getattribute__(s, '_callmethod')(attr))
    MyManager.register(name, cls, proxy)

register_proxy('test', TestClass, TestProxy)
register_proxy('another', AnotherClass, AnotherProxy)

if __name__ == '__main__':
    manager = MyManager()
    manager.start()
    mt = manager.test(2)
    ma = manager.another(3)
    mt.b()
    ma.c()
    mt.a = 5
    ma.a = 6
    mt.b()
    ma.c()

Answer 2

在花了几个小时阅读源代码之后，以下是实现代理类以公开所有属性和方法的最简单方法：

class TestProxy(NamespaceProxy):
    _exposed_ = tuple(dir(Test))

    def __getattr__(self, name):
        result = super().__getattr__(name)
        if isinstance(result, types.MethodType):
            def wrapper(*args, **kwargs):
                self._callmethod(name, args)
            return wrapper
        return result

BaseManager.register('Test', Test, TestProxy)

manager = BaseManager()
test = manager.Test()

另外，这是一个自动代理方法：

def Proxy(target):
    dic = {'types': types}
    exec('''def __getattr__(self, key):
        result = self._callmethod('__getattribute__', (key,))
        if isinstance(result, types.MethodType):
            def wrapper(*args, **kwargs):
                self._callmethod(key, args)
            return wrapper
        return result''', dic)
    proxyName = target.__name__ + "Proxy"
    ProxyType = type(proxyName, (NamespaceProxy,), dic)
    ProxyType._exposed_ = tuple(dir(target))
    return ProxyType

TestProxy = Proxy(Test)
BaseManager.register('Test', Test, TestProxy)

manager = BaseManager()
test = manager.Test()