在Boost Python中展示一个非const但不可复制的成员

Question

这是我的问题:

我有两个这样的课程:

class Signal {
public:
    void connect(...) { sig.connect(...); }
private:
    boost::signal2::signal sig;
};

class MyClass {
public:
    Signal on_event;
};

我想暴露,MyClass::on_event以便我可以my_class_instance.on_event.connect(...)从Python 调用.

这就是我如何包装这些类:

class_<Signal, boost::noncopyable> ("Signal", noinit)
    .def("connect", &some_helper_function);

class_<MyClass> ("MyClass")
    .def_readonly("on_event", &MyClass::on_event);

这编译,但当我尝试connect从Python 调用时,我得到:AttributeError: can't set attribute.这在这里解释:http://www.boost.org/doc/libs/1_53_0/libs/python/doc/tutorial/doc/html/python/exposing.html,所以我改为.def_readwritefor on_event.

但现在我得到了编译时错误.它几乎不可能读取C++模板错误消息,但据我所知,因为它boost::signals2::signal是不可复制的.由于.def_readwrite使成员可分配,因此它不能是不可复制的.但对于我的用法我不想分配成员,我只是不想调用一个方法.

我考虑制作const 的connect方法Signal,即使它改变了对象,但是我不能sig.connect()从那个方法调用,所以这是一个不行的...

有任何想法吗？

Answer 1

我在重现您的结果时遇到了问题,但这里有一些可能有助于解决问题的信息.

简单的课程:

class Signal
{
public:
  void connect() { std::cout << "connect called" << std::endl; }
private:
  boost::signals2::signal<void()> signal_;
};

class MyClass
{
public:
  Signal on_event;
};

和基本绑定:

namespace python = boost::python;
python::class_<Signal, boost::noncopyable>("Signal", python::no_init)
  .def("connect", &Signal::connect)
  ;

python::class_<MyClass>("MyClass")
  .def_readonly("on_event", &MyClass::on_event)
  ;

代码无法编译.暴露类时,Boost.Python的默认行为是注册转换器.这些转换器需要复制构造函数,作为将C++类对象复制到可由Python对象管理的存储中的方法.通过提供类型boost::noncopyable的参数,可以为类禁用此行为class_.

在这种情况下,MyClass绑定不会抑制复制构造函数.Boost.Python将尝试在绑定中使用复制构造函数,并因编译器错误而失败,因为成员变量on_event不可复制. Signal是不可复制的,因为它包含一个boost::signal2::signal继承自的类型的成员变量boost::noncopyable.

boost:::noncopyable作为参数类型添加到MyClass绑定允许编译代码.

namespace python = boost::python;
python::class_<Signal, boost::noncopyable>("Signal", python::no_init)
  .def("connect", &Signal::connect)
  ;

python::class_<MyClass, boost::noncopyable>("MyClass")
  .def_readonly("on_event", &MyClass::on_event)
  ;

用法:

>>> import example
>>> m = example.MyClass()
>>> m.on_event.connect()
connect called
>>> m.on_event = None
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
>>> 

虽然此设置允许所需的绑定和调用语法,但看起来它似乎是最终目标的第一步.

My apologies if this is too presumptuous. However, based on other recent questions, I would like to take the time to expand upon the initial example to cover what appears to be the final goal: being able to connect Python callbacks to a signal2::signal. I will cover two different approaches, as the mechanics and level of complexity differ, but they may provide insight into details that should be considered for the final solution.

Only Python threads.

For this first scenario, lets assume that only Python threads are interacting with the library.

One technique that keeps it relatively simple is to use inheritance. Start by defining a helper Slot class that can connect to Signal.

class Slot
  : public boost::python::wrapper<Slot>
{
public:
  void operator()()
  {
    this->get_override("__call__")();
  }
};

的Slot类所继承boost::python::wrapper,即unintrusively提供了钩子以允许Python类覆盖在基类函数的类.

当可调用类型连接到时boost::signals2::signal,信号可以将参数复制到其内部列表中.因此,对于仿函数Slot来说,只要它保持连接就能够延长实例的寿命是很重要的signal.实现这一目标的最简单方法是通过管理Slot来实现boost::shared_ptr.

结果Signal类看起来像:

class Signal
{
public:
  template <typename Callback>
  void connect(const Callback& callback)
  {
    signal_.connect(callback);
  }

  void operator()() { signal_(); }
private:
  boost::signals2::signal<void()> signal_;
};

并且辅助函数有助于保持Signal::connect通用,以防其他C++类型需要连接到它.

void connect_slot(Signal& self, 
                  const boost::shared_ptr<Slot>& slot)
{
  self.connect(boost::bind(&Slot::operator(), slot));
}

这导致以下绑定:

BOOST_PYTHON_MODULE(example) {
  namespace python = boost::python;
  python::class_<Signal, boost::noncopyable>("Signal", python::no_init)
    .def("connect",  &connect_slot)
    .def("__call__", &Signal::operator())
    ;

  python::class_<MyClass, boost::noncopyable>("MyClass")
    .def_readonly("on_event", &MyClass::on_event)
    ;

  python::class_<Slot, boost::shared_ptr<Slot>, 
                 boost::noncopyable>("Slot")
    .def("__call__", python::pure_virtual(&Slot::operator()))
    ;
}

它的用法如下:

>>> from example import *
>>> class Foo(Slot):
...     def __call__(self):
...          print "Foo::__call__"
... 
>>> m = MyClass()
>>> foo = Foo()
>>> m.on_event.connect(foo)
>>> m.on_event()
Foo::__call__
>>> foo = None
>>> m.on_event()
Foo::__call__

虽然成功,但它具有不是pythonic的不幸特征.例如:

>>> def spam():
...     print "spam"
... 
>>> m = MyClass()
>>> m.on_event.connect(spam)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Boost.Python.ArgumentError: Python argument types in
    Signal.connect(Signal, function)
did not match C++ signature:
    connect(Signal {lvalue}, boost::shared_ptr<Slot>)

如果任何可调用对象可以连接到信号,那将是理想的.一个简单的方法就是在Python中修补绑定.为了对最终用户透明:

• 将C++绑定模块名称从更改example为_example.确保还更改库名称.
• 创建example.py将修补Signal.connect()以将参数包装为继承自的类型Slot.

example.py 可能看起来像这样:

from _example import *

class _SlotWrap(Slot):

    def __init__(self, fn):
        self.fn = fn
        Slot.__init__(self)

    def __call__(self):
        self.fn()

def _signal_connect(fn):
    def decorator(self, slot):
        # If the slot is not an instance of Slot, then aggregate it
        # in SlotWrap.
        if not isinstance(slot, Slot):
            slot = _SlotWrap(slot)
        # Invoke the decorated function with the slot.
        return fn(self, slot)
    return decorator

# Patch Signal.connect.
Signal.connect = _signal_connect(Signal.connect)

修补程序对最终用户是无缝的.

>>> from example import *
>>> def spam():
...     print "spam"
... 
>>> m = MyClass()
>>> m.on_event.connect(spam)
>>> m.on_event()
spam

使用此补丁,任何可调用类型都可以连接,Signal而无需显式继承Slot.因此,它比初始解决方案变得更加pythonic.永远不要低估保持绑定简单和非pythonic的好处,但在python中修补它们是pythonic.

Python和C++线程.

In the next scenario, lets consider the case where C++ threads are interacting with Python. For example, a C++ thread can be set to invoke the signal after a period of time.

This example can become fairly involved, so lets start with the basics: Python's Global Interpreter Lock (GIL). In short, the GIL is a mutex around the interpreter. If a thread is doing anything that affects reference counting of python managed object, then it needs to have acquired the GIL. In the previous example, as there were no C++ threads, all actions occurred while the GIL had been acquired. While this fairly straightforward, it can become complex rather quickly.

First, the module needs to have Python initialize the GIL for threading.

BOOST_PYTHON_MODULE(example) {
  PyEval_InitThreads(); // Initialize GIL to support non-python threads.
  ...
}

For convenience, lets create a simple class to help manage the GIL:

/// @brief RAII class used to lock and unlock the GIL.
class gil_lock
{
public:
  gil_lock()  { state_ = PyGILState_Ensure(); }
  ~gil_lock() { PyGILState_Release(state_);   }
private:
  PyGILState_STATE state_;
};

The thread will be invoking MyClass's signal. Thus, it needs to extend the lifetime of MyClass while the thread is alive. A good candidate to accomplish this is by managing MyClass with a shared_ptr.

Lets identify when the C++ thread will need the GIL:

• MyClass being deleted by shared_ptr.
• boost::signals2::signal can make additional copies of connected objects, as is done when the signal is concurrently invoked.
• Invoking a Python objected connected through boost::signals2::signal. The callback will certainly affect python objects. For example, the self argument provided to the __call__ method will increase and decrease an object's reference count.

Supporting MyClass being deleted from a C++ thread.

To guarantee the GIL is held when MyClass is deleted by shared_ptr from within a C++ thread, a custom deleter is required. This also requires the bindings to suppress the default constructor, and use a custom constructor instead.

/// @brief Custom deleter.
template <typename T>
struct py_deleter
{
  void operator()(T* t)
  {
    gil_lock lock;    
    delete t;
  }
};

/// @brief Create Signal with a custom deleter.
boost::shared_ptr<MyClass> create_signal()
{
  return boost::shared_ptr<MyClass>(
    new MyClass(),
    py_deleter<MyClass>());
}

...

BOOST_PYTHON_MODULE(example) {

  ...

  python::class_<MyClass, boost::shared_ptr<MyClass>,
                 boost::noncopyable>("MyClass", python::no_init)
    .def("__init__", python::make_constructor(&create_signal))
    .def_readonly("on_event", &MyClass::on_event)
    ;
}

The thread itself.

The thread's functionality is fairly basic: it sleeps then invokes the signal. However, it is important to understand the context of the GIL.

/// @brief Wait for a period of time, then invoke the
///        signal on MyClass.
void call_signal(boost::shared_ptr<MyClass>& shared_class,
                 unsigned int seconds)
{
  // The shared_ptr was created by the caller when the GIL was
  // locked, and is accepted as a reference to avoid modifying
  // it while the GIL is not locked.

  // Sleep without the GIL so that other python threads are able
  // to run.
  boost::this_thread::sleep_for(boost::chrono::seconds(seconds));

  // We do not want to hold the GIL while invoking C++-specific
  // slots connected to the signal.  Thus, it is the responsibility of
  // python slots to lock the GIL.  Additionally, the potential
  // copying of slots internally by the signal will be handled through
  // another mechanism.
  shared_class->on_event();

  // The shared_class has a custom deleter that will lock the GIL
  // when deletion needs to occur.
}

/// @brief Function that will be exposed to python that will create
///        a thread to call the signal.
void spawn_signal_thread(boost::shared_ptr<MyClass> self,
                       unsigned int seconds)
{
  // The caller owns the GIL, so it is safe to make copies.  Thus,
  // spawn off the thread, binding the arguments via copies.  As
  // the thread will not be joined, detach from the thread.
  boost::thread(boost::bind(&call_signal, self, seconds)).detach();
}

And the MyClass bindings are updated.

python::class_<MyClass, boost::shared_ptr<MyClass>,
               boost::noncopyable>("MyClass", python::no_init)
  .def("__init__", python::make_constructor(&create_signal))
  .def("signal_in", &spawn_signal_thread)
  .def_readonly("on_event", &MyClass::on_event)
  ;

boost::signals2::signal interacting with python objects.

boost::signals2::signal may make copies when it is invoked. Additionally, there may be C++ slots connected to the signal, so it would be ideal to not have the GIL locked while the signal is invoked. However, signal does not provide hooks to allow us to acquire the GIL before creating copies of slots or invoking the slot.

To add to the complexity, when the bindings expose a C++ function that accepts a C++ class with a HeldType that is not a smart pointer, then Boost.Python will extract the non-reference counted C++ object from the reference-counted python object. It can safely do this because the calling thread, in Python, has the GIL. To maintain a reference count to slots trying to connect from Python, as well as allow for any callable type to connect, we can use the opaque type of boost::python::object.

In order to avoid having signal create copies of the provided boost::python::object, one can create a copy of boost::python::object so that reference counts remain accurate, and manage the copy via shared_ptr. This allows signal to freely create copies of shared_ptr instead of creating boost::python::object without the GIL.

This GIL safety slot can be encapsulated in a helper class.

/// @brief Helper type that will manage the GIL for a python slot.
class py_slot
{
public:

  /// @brief Constructor that assumes the caller has the GIL locked.
  py_slot(const boost::python::object& object)
    : object_(new boost::python::object(object),   // GIL locked, so copy.
              py_deleter<boost::python::object>()) // Delete needs GIL.
  {}

  void operator()()
  {
    // Lock the gil as the python object is going to be invoked.
    gil_lock lock;
    (*object_)(); 
  }

private:
  boost::shared_ptr<boost::python::object> object_;
};

A helper function will be exposed to Python to help adapt the types.

/// @brief Signal connect helper.
void signal_connect(Signal& self,
                    boost::python::object object)
{
  self.connect(boost::bind(&py_slot::operator(), py_slot(object)));
}

And the updated binding expose the helper function:

python::class_<Signal, boost::noncopyable>("Signal", python::no_init)
  .def("connect",  &signal_connect)
  .def("__call__", &Signal::operator())
  ;

The final solution looks like this:

#include <boost/bind.hpp>
#include <boost/python.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/signals2/signal.hpp>
#include <boost/thread.hpp>

class Signal
{
public:
  template <typename Callback>
  void connect(const Callback& callback)
  {
    signal_.connect(callback);
  }

  void operator()() { signal_(); }
private:
  boost::signals2::signal<void()> signal_;
};

class MyClass
{
public:
  Signal on_event;
};

/// @brief RAII class used to lock and unlock the GIL.
class gil_lock
{
public:
  gil_lock()  { state_ = PyGILState_Ensure(); }
  ~gil_lock() { PyGILState_Release(state_);   }
private:
  PyGILState_STATE state_;
};    

/// @brief Custom deleter.
template <typename T>
struct py_deleter
{
  void operator()(T* t)
  {
    gil_lock lock;    
    delete t;
  }
};

/// @brief Create Signal with a custom deleter.
boost::shared_ptr<MyClass> create_signal()
{
  return boost::shared_ptr<MyClass>(
    new MyClass(),
    py_deleter<MyClass>());
}

/// @brief Wait for a period of time, then invoke the
///        signal on MyClass.
void call_signal(boost::shared_ptr<MyClass>& shared_class,
                 unsigned int seconds)
{
  // The shared_ptr was created by the caller when the GIL was
  // locked, and is accepted as a reference to avoid modifying
  // it while the GIL is not locked.

  // Sleep without the GIL so that other python threads are able
  // to run.
  boost::this_thread::sleep_for(boost::chrono::seconds(seconds));

  // We do not want to hold the GIL while invoking C++-specific
  // slots connected to the signal.  Thus, it is the responsibility of
  // python slots to lock the GIL.  Additionally, the potential
  // copying of slots internally by the signal will be handled through
  // another mechanism.
  shared_class->on_event();

  // The shared_class has a custom deleter that will lock the GIL
  // when deletion needs to occur.
}

/// @brief Function that will be exposed to python that will create
///        a thread to call the signal.
void spawn_signal_thread(boost::shared_ptr<MyClass> self,
                       unsigned int seconds)
{
  // The caller owns the GIL, so it is safe to make copies.  Thus,
  // spawn off the thread, binding the arguments via copies.  As
  // the thread will not be joined, detach from the thread.
  boost::thread(boost::bind(&call_signal, self, seconds)).detach();
}

/// @brief Helepr type that will manage the GIL for a python slot.
struct py_slot
{
public:

  /// @brief Constructor that assumes the caller has the GIL locked.
  py_slot(const boost::python::object& object)
    : object_(new boost::python::object(object),   // GIL locked, so copy.
              py_deleter<boost::python::object>()) // Delete needs GIL.
  {}

  void operator()()
  {
    // Lock the gil as the python object is going to be invoked.
    gil_lock lock;
    (*object_)(); 
  }

private:
  boost::shared_ptr<boost::python::object> object_;
};

/// @brief Signal connect helper.
void signal_connect(Signal& self,
                    boost::python::object object)
{
  self.connect(boost::bind(&py_slot::operator(), py_slot(object)));
}

BOOST_PYTHON_MODULE(example) {
  PyEval_InitThreads(); // Initialize GIL to support non-python threads.

  namespace python = boost::python;
  python::class_<Signal, boost::noncopyable>("Signal", python::no_init)
    .def("connect",  &signal_connect)
    .def("__call__", &Signal::operator())
    ;

  python::class_<MyClass, boost::shared_ptr<MyClass>,
                 boost::noncopyable>("MyClass", python::no_init)
    .def("__init__", python::make_constructor(&create_signal))
    .def("signal_in", &spawn_signal_thread)
    .def_readonly("on_event", &MyClass::on_event)
    ;
}

And a testing script (test.py):

from time import sleep
from example import *

def spam():
    print "spam"

m = MyClass()
m.on_event.connect(spam)
m.on_event()

m.signal_in(2)
m = None
print "Sleeping"
sleep(5)
print "Done sleeping"

Results in the following:

spam
Sleeping
spam
Done sleeping

In conclusion, when an object is passed through the Boost.Python layer, take time to consider how to manage its lifespan and the context in which it will be used. This often requires understanding how other libraries being used will handle the object. It is not an easy problem, and providing a pythonic solution can be a challenge.