在C中嵌入Python:尝试在Python代码调用的C回调中调用Python代码时出错

Question

我有一些调用Python函数的C代码.这个Python函数接受一个地址,并使用WINFUNCTYPE最终将它转换为Python可以调用的函数.C函数作为参数发送到Python函数最终将调用另一个Python函数.正是在最后一步导致崩溃.所以简而言之,我从C - > Python - > C - > Python.最后一个C - > Python导致崩溃.我一直试图理解这个问题,但我一直无法理解.

有人可以指出我的问题吗？

使用Visual Studio 2010编译的C代码,并使用args"c:\ ...\crash.py"和"func1"运行:

#include <stdlib.h>
#include <stdio.h>

#include <Python.h>

PyObject* py_lib_mod_dict; //borrowed

void __stdcall cfunc1()
{
    PyObject* py_func;
    PyObject* py_ret;
    int size;
    PyGILState_STATE gil_state;

    gil_state = PyGILState_Ensure();
    printf("Hello from cfunc1!\n");

    size = PyDict_Size(py_lib_mod_dict);
    printf("The dictionary has %d items!\n", size);
    printf("Calling with GetItemString\n");
    py_func = PyDict_GetItemString(py_lib_mod_dict, "func2"); //fails here when cfunc1 is called via callback... will not even go to the next line!
    printf("Done with GetItemString\n");
    py_ret = PyObject_CallFunction(py_func, 0);

    if (py_ret)
    {
        printf("PyObject_CallFunction from cfunc1 was successful!\n");
        Py_DECREF(py_ret);
    }
    else
        printf("PyObject_CallFunction from cfunc1 failed!\n");

    printf("Goodbye from cfunc1!\n");
    PyGILState_Release(gil_state);
}

int wmain(int argc, wchar_t** argv)
{
    PyObject* py_imp_str;
    PyObject* py_imp_handle;
    PyObject* py_imp_dict; //borrowed
    PyObject* py_imp_load_source; //borrowed
    PyObject* py_dir; //stolen
    PyObject* py_lib_name; //stolen
    PyObject* py_args_tuple;
    PyObject* py_lib_mod;
    PyObject* py_func;
    PyObject* py_ret;

    Py_Initialize();

    //import our python script
    py_dir = PyUnicode_FromWideChar(argv[1], wcslen(argv[1]));
    py_imp_str = PyString_FromString("imp");
    py_imp_handle = PyImport_Import(py_imp_str);
    py_imp_dict = PyModule_GetDict(py_imp_handle); //borrowed
    py_imp_load_source = PyDict_GetItemString(py_imp_dict, "load_source"); //borrowed
    py_lib_name = PyUnicode_FromWideChar(argv[2], wcslen(argv[2]));

    py_args_tuple = PyTuple_New(2);
    PyTuple_SetItem(py_args_tuple, 0, py_lib_name); //stolen
    PyTuple_SetItem(py_args_tuple, 1, py_dir); //stolen

    py_lib_mod = PyObject_CallObject(py_imp_load_source, py_args_tuple);
    py_lib_mod_dict = PyModule_GetDict(py_lib_mod); //borrowed

    printf("Calling cfunc1 from main!\n");
    cfunc1();

    py_func = PyDict_GetItem(py_lib_mod_dict, py_lib_name);
    py_ret = PyObject_CallFunction(py_func, "(I)", &cfunc1);

    if (py_ret)
    {
        printf("PyObject_CallFunction from wmain was successful!\n");
        Py_DECREF(py_ret);
    }
    else
        printf("PyObject_CallFunction from wmain failed!\n");

    Py_DECREF(py_imp_str);
    Py_DECREF(py_imp_handle);
    Py_DECREF(py_args_tuple);
    Py_DECREF(py_lib_mod);

    Py_Finalize();

    fflush(stderr);
    fflush(stdout);
    return 0;
}

Python代码:

from ctypes import *

def func1(cb):
    print "Hello from func1!"
    cb_proto = WINFUNCTYPE(None)
    print "C callback: " + hex(cb)
    call_me = cb_proto(cb)
    print "Calling callback from func1."
    call_me()
    print "Goodbye from func1!"

def func2():
    print "Hello and goodbye from func2!"

输出:

Calling cfunc1 from main!
Hello from cfunc1!
The dictionary has 88 items!
Calling with GetItemString
Done with GetItemString
Hello and goodbye from func2!
PyObject_CallFunction from cfunc1 was successful!
Goodbye from cfunc1!
Hello from func1!
C callback: 0x1051000
Calling callback from func1.
Hello from cfunc1!
The dictionary has 88 items!
Calling with GetItemString
PyObject_CallFunction from wmain failed!

我在最后添加了一个PyErr_Print(),结果如下:

Traceback (most recent call last):
  File "C:\Programming\crash.py", line 9, in func1
    call_me()
WindowsError: exception: access violation writing 0x0000000C

编辑:修正了abarnert指出的错误.输出不受影响.
编辑:在解决该错误的代码中添加(在cfunc1中获取GIL锁).再次感谢abarnert.

Answer 1

问题是这段代码:

py_func = PyDict_GetItemString(py_lib_mod_dict, "func2"); //fails here when cfunc1 is called via callback... will not even go to the next line!
printf("Done with GetItemString\n");
py_ret = PyObject_CallFunction(py_func, 0);

Py_DECREF(py_func);

正如文档所说,PyDict_GetItemString返回借用的引用.因此,当你第一次在这里打电话时,你借用了引用,然后减去它,导致它被销毁.下次你打电话时,你会回来垃圾,并试着打电话给它.

因此,要修复它,只需删除Py_DECREF(py_func)(或在行Py_INCREF(py_func)后添加pyfunc =).

其实,你通常会得到一个特殊的"死亡"对象,这样你就可以测试这个漂亮容易:把PyObject_Print(py_func, stdout)后py_func =线和后Py_DECREF线,你可能会看到类似<function func2 at 0x10b9f1230>的第一次,<refcnt 0 at 0x10b9f1230>第二次和第三次(你不会看到第四个,因为它会在你到达之前崩溃.

我没有Windows中方便,但不断变化的wmain,wchar_t,PyUnicode_FromWideChar,WINFUNCTYPE,等至main,char,PyString_FromString,CFUNCTYPE,等,我是能够建立并运行代码,和我在同一个地方崩溃......和修复工程.

另外......你不应该把GIL放在里面cfunc1吗？我不经常写这样的代码,所以也许我错了.而且我不会因为代码原因而崩溃.产生一个运行的线程显然cfunc1 会崩溃,并且PyGILState_Ensure/ Release解决了崩溃......但这并不能证明你在单线程的情况下需要任何东西.所以也许这是不相关的...但是如果你在修复第一个之后再次崩溃(在线程的情况下,我看起来像Fatal Python error: PyEval_SaveThread: NULL tstate),请看看这个.

顺便说一句,如果你是Python扩展和嵌入的新手:大量的无法解释的崩溃,就像这个,由手动引用计数错误引起的.这就是boost::python存在等等的原因.并不是说用普通的C API做到这一点是不可能的,只是它很容易弄错,而且你将不得不习惯于调试这样的问题.