sda*_*aau 24 python debugging gdb
假设我们有以下超级简单的Python脚本:
print "Initializing"....
a=10
print "Variable value is %d" % (a)
print "All done!"
...并且说,我想通过在线上放置一个断点来调试这个脚本a=10,然后单步调试脚本.
现在,我想使用gdb它,因为我想调试可能作为共享对象(.so)库的一部分的Python绑定- 因此,我理想地在Python代码行上放置一个断点,并且然后"进入"共享对象的C部分...(注意,DebuggingWithGdb - PythonInfo Wiki并没有真正明确说明这是可能的)
问题是:gdb它本身无法真正识别断点,放在Python脚本行上:
$ gdb python
GNU gdb (GDB) 7.3.50.20110806-cvs
...
Reading symbols from /usr/bin/python...(no debugging symbols found)...done.
(gdb) b test.py:3
No symbol table is loaded. Use the "file" command.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (test.py:3) pending.
(gdb) run test.py
Starting program: /usr/bin/python test.py
...
...当整个Python脚本在其中运行时gdb,断点就永远不会到达.
所以 - 就是我想做的事情,尽可能的gdb; 如果没有,我会对类似的东西有什么其他选择？
Mic*_*man 26
非常有趣的问题.这是我的方法.创建signal_test.py:
import os
import signal
PID = os.getpid()
def do_nothing(*args):
pass
def foo():
print "Initializing..."
a=10
os.kill(PID, signal.SIGUSR1)
print "Variable value is %d" % (a)
print "All done!"
signal.signal(signal.SIGUSR1, do_nothing)
foo()
然后你可以在gdb下运行它:
$ gdb --args python signal_test.py
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-37.el5_7.1)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /net/gs/vol3/software/modules-sw/python/2.7/Linux/RHEL5/x86_64/bin/python...done.
当你运行它时,它会一直运行到你打电话给kill():
(gdb) run
Starting program: /net/gs/vol3/software/modules-sw/python/2.7/Linux/RHEL5/x86_64/bin/python signal_test.py
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x2aaaaaaab000
[Thread debugging using libthread_db enabled]
Initializing...
Program received signal SIGUSR1, User defined signal 1.
0x0000003d340306f7 in kill () from /lib64/libc.so.6
然后你可以看一个回溯:
(gdb) backtrace
#0 0x0000003d340306f7 in kill () from /lib64/libc.so.6
#1 0x00000000004d82dd in posix_kill (self=<value optimized out>, args=<value optimized out>)
at ./Modules/posixmodule.c:4047
#2 0x000000000049b574 in call_function (f=0x8aca30, throwflag=<value optimized out>)
at Python/ceval.c:4012
#3 PyEval_EvalFrameEx (f=0x8aca30, throwflag=<value optimized out>) at Python/ceval.c:2665
#4 0x000000000049c5cd in call_function (f=0x8ac560, throwflag=<value optimized out>)
at Python/ceval.c:4098
#5 PyEval_EvalFrameEx (f=0x8ac560, throwflag=<value optimized out>) at Python/ceval.c:2665
#6 0x000000000049d3bb in PyEval_EvalCodeEx (co=0x2aaaae224f30, globals=<value optimized out>,
locals=<value optimized out>, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0,
closure=0x0) at Python/ceval.c:3252
#7 0x000000000049d432 in PyEval_EvalCode (co=0x1a48, globals=0xa, locals=0x0) at Python/ceval.c:666
#8 0x00000000004bf321 in run_mod (fp=0x89ad60, filename=0x7fffffffb5b4 "signal_test.py",
start=<value optimized out>, globals=0x7e4680, locals=0x7e4680, closeit=1, flags=0x7fffffffaee0)
at Python/pythonrun.c:1346
#9 PyRun_FileExFlags (fp=0x89ad60, filename=0x7fffffffb5b4 "signal_test.py",
start=<value optimized out>, globals=0x7e4680, locals=0x7e4680, closeit=1, flags=0x7fffffffaee0)
at Python/pythonrun.c:1332
#10 0x00000000004bf5d8 in PyRun_SimpleFileExFlags (fp=<value optimized out>,
filename=0x7fffffffb5b4 "signal_test.py", closeit=1, flags=0x7fffffffaee0)
at Python/pythonrun.c:936
#11 0x00000000004148cc in Py_Main (argc=<value optimized out>, argv=<value optimized out>)
at Modules/main.c:599
#12 0x0000003d3401d994 in __libc_start_main () from /lib64/libc.so.6
#13 0x0000000000413b19 in _start ()
如果继续,程序的其余部分将正常运行.
(gdb) continue
Continuing.
Variable value is 10
All done!
Program exited normally.
相反,您可以在适当的框架中逐步执行,直到达到您感兴趣的语句.您可能希望运行调试Python以使其更有意义.
sda*_*aau 22
为长篇职位道歉; 我再次回到类似的调试问题 - 你需要长途跋涉到调试器,最后发现没有实际的错误 - 所以我只想在这里发布我的笔记和一些代码(我是仍然在Python 2.7,Ubuntu 11.04).关于OP问题 - 在更新gdb的问题中,它也可以通过使用id(...)Python脚本中的函数来中断,并且可以gdb中断builtin_id; 但是这里有更多细节:
再次,我遇到了一个用于Python的C .so共享库模块的问题; 这次是svn.client,这是一个Swig模块(另见这里); 在Debian/Ubuntu中可用sudo apt-get install python-subversion(文件列表).尝试运行示例8.3时出现问题.Python状态爬虫 - 使用API(svnbook)此示例应该与terminal命令相同svn status; 但是当我在我的一个工作副本上尝试它时,它崩溃了" 错误(22):将目录'路径'中的条目转换为UTF-8时出错 ",即使svn status已处理相同的工作副本(WC)目录(对于现在几年) - 所以我想看看它来自哪里.我的测试脚本版本是python-subversion-test.py; 我的完整调试日志是在logsvnpy.gz(gzip文件文件,~188K未压缩,如果有人想通过无休止的步进和回溯趟过) - 这是删节版本.我安装了Python 2.7和3.2,但默认情况下2.7是Ubuntu 11.04:
$ ls -la $(which python python-dbg)
lrwxrwxrwx 1 root root 9 2012-02-29 07:31 /usr/bin/python -> python2.7
lrwxrwxrwx 1 root root 13 2013-04-07 03:01 /usr/bin/python-dbg -> python2.7-dbg
$ apt-show-versions -r 'python[^-]+'
libpython2.7/natty uptodate 2.7.1-5ubuntu2.2
libpython3.2/natty uptodate 3.2-1ubuntu1.2
python2.7/natty uptodate 2.7.1-5ubuntu2.2
python2.7-dbg/natty uptodate 2.7.1-5ubuntu2.2
python2.7-dev/natty uptodate 2.7.1-5ubuntu2.2
python2.7-minimal/natty uptodate 2.7.1-5ubuntu2.2
python3/natty uptodate 3.2-1ubuntu1
python3-minimal/natty uptodate 3.2-1ubuntu1
python3.2/natty uptodate 3.2-1ubuntu1.2
python3.2-minimal/natty uptodate 3.2-1ubuntu1.2
首先要注意的是Python示例如何运行:在那里,为了获取目录中所有文件的状态,首先svn.client.svn_client_status2调用 - 除了路径,还在_status_callback参数中,作为Python中的回调函数进行注册 -然后阻止.在status2阻塞时,底层模块遍历WC目录路径中的所有文件; 并且对于每个文件条目,它调用已注册的_status_callback应该打印出关于该条目的信息.一旦这个递归结束,status2退出.因此,UTF-8故障必须来自底层模块.进一步检查此模块:
$ python -c 'import inspect,pprint,svn.client; pprint.pprint(inspect.getmembers(svn.client))' | grep status
('status', <function svn_client_status at 0xb7351f44>),
('status2', <function svn_client_status2 at 0xb7351f0c>),
('status3', <function svn_client_status3 at 0xb7351ed4>),
('status4', <function svn_client_status4 at 0xb7351e9c>),
('svn_client_status', <function svn_client_status at 0xb7351f44>),
# ...
...显示还有其他statusX功能 - 但是,status3失败并出现相同的UTF-8错误; 而status4导致分段错误(这成为调试的另一个问题).
再次,正如我对@EliBendersky的回答一样,我想在Python中发出一个断点,以便稍后获得某种C函数的调用堆栈,这将揭示问题发生的位置 - 没有我得到从源头重建C模块; 但事实并非如此简单.
首先,可能非常混乱的一件事是gdb与Python 之间的关系; 这里出现的典型资源是:
gdbinit在"GDB宏"中提到了一个,也就是说 release27-MAINT /杂项/ gdbinit在Python源树; 定义gdb像pylocals和的命令pyframe,但也提到:
#注意:如果您有gdb 7或更高版本,它支持直接调试Python
#使用嵌入式宏,您可能会发现它优于此处的内容.
#请参阅Tools/gdb/libpython.py和http://bugs.python.org/issue8032.
功能/ EasierPythonDebugging - FedoraProject - 有一个例子,提到了Fedora python-debuginfo包,和libpython
从gdb 7开始,可以配置gdb的构建--with-python,允许
使用Python代码扩展gdb ,例如用于特定于库的数据可视化,
例如C++ STL类型.....
这个模块嵌入了有关libpython的实现细节的知识,以便
我们可以发出有用的可视化,例如字符串,列表,字典,
给出文件/行信息的框架和局部变量的状态
gdbPython的功能这有点令人困惑 - 除了指针之外,最好让自己获得gdbv.7; 我设法得到我的操作系统:
$ apt-show-versions gdb
gdb 7.3-50.20110806-cvs newer than version in archive
测试是否gdb支持Python的快速方法是:
$ gdb --batch --eval-command="python print gdb"
<module 'gdb' (built-in)>
$ python -c 'import gdb; print gdb'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named gdb
...但gdb支持Python,并不意味着Python本身可以访问gdb功能(显然,它gdb有自己内置的独立Python解释器).
事实证明,在Ubuntu 11.04中,python2.7-dbg软件包安装了一个文件libpython2.7.so.1.0-gdb.py:
$ find / -xdev -name '*libpython*' 2>/dev/null | grep '\.py'
/usr/lib/debug/usr/lib/libpython2.7.so.1.0-gdb.py
$ sudo ln -s /usr/lib/debug/usr/lib/libpython2.7.so.1.0-gdb.py /usr/lib/debug/usr/lib/libpython.py
......这是与上述相对应的Tools/gdb/libpython.py; 符号链接将允许我们将其称为libpython,并使用Features/EasierPythonDebugging中提到的导入脚本 .
该test_gdb.py脚本实际上是针对Python 3的 - 我已经将其修改为2.7,并发布在test_gdb2.7.py中.该脚本gdb通过OS系统调用调用,并测试其Python功能,打印输出到stdout; 它还接受命令行选项,-imp-lp该选项将import libpython在gdb执行其他命令之前执行.所以,例如:
$ python-dbg test_gdb2.7.py
...
*** test_prettyprint ***
42 (self=0x0, v=0x8333fc8)
[] (self=0x0, v=0xb7f7506c)
('foo', 'bar', 'baz') (self=0x0, v=0xb7f7d234)
[0, 1, 2, 3, 4] (self=0x0, v=0xb7f7506c)
...
$ python-dbg test_gdb2.7.py -imp-lp
...
*** test_prettyprint ***
42 (self=0x0, v=42)
[] (self=0x0, v=[])
('foo', 'bar', 'baz') (self=0x0, v=('foo', 'bar', 'baz'))
[0, 1, 2, 3, 4] (self=0x0, v=[0, 1, 2, 3, 4])
...
因此,libpython.py它专门用于内部的Python解释器gdb,它有助于gdb打印Python表示(v=[])而不仅仅是内存地址(v=0xb7f7506c) - 这只是有用的,如果 gdb调试Python脚本(或者更确切地说,它将调试Python可执行文件,解释脚本).
The test_gdb.py script also gives the pointer that you can "... run "python -c'id(DATA)'" under gdb with a breakpoint on builtin_id"; for testing this, I have posted a bash script, gdb_py_so_test.sh, which creates an executable with a counting thread function, and both plain distutils and swig modules (in both debug and release versions) that interface to the same function. It also creates a .gdbinit with both gdb and gdb's Python class breakpoints - and finally it runs gdb on Python (loading one of the shared modules), where the user can hopefully see if the breakpoints are really triggering.
First I focused on the status4 segfault, and I wanted to know exactly which module does the function come from. I used a function, that can be found in debug_funcs.py; which can be called with separate regex for functions and modules, and may generate something like:
$ python python-subversion-test.py ./MyRepoWCDir
# ...
# example for debug_funcs.showLoadedModules(r'(?=.*\.(so|pyc))(?=.*svn)(?=.*client)')
#
svn.client 0xb74b83d4L <module 'svn.client' from '/usr/lib/pymodules/python2.7/svn/client.pyc'>
_client 0xb7415614L <module '_client' from '/usr/lib/pymodules/python2.7/libsvn/_client.so'>
libsvn.client 0xb74155b4L <module 'libsvn.client' from '/usr/lib/pymodules/python2.7/libsvn/client.pyc'>
#
# example for debug_funcs.showFunctionsInLoadedModules(r'status4', r'(?=.*\.(so|pyc))(?=.*svn)')
#
0xb738c4fcL libsvn.client svn_client_status4 libsvn/client.pyc
0xb74e9eecL _client svn_client_status4 libsvn/_client.so
0xb738c4fcL svn.client status4 svn/client.pyc
0xb738c4fcL svn.client svn_client_status4 svn/client.pyc
However, note that:
$ python-dbg python-subversion-test.py ./MyRepoWCDir
# ...
0x90fc574 - _client /usr/lib/pymodules/python2.7/libsvn/_client_d.so
# ...
0x912b30c _client svn_client_status4 libsvn/_client_d.so
# ...
$ apt-show-versions -r python-subversion
python-subversion/natty uptodate 1.6.12dfsg-4ubuntu2.1
python-subversion-dbg/natty uptodate 1.6.12dfsg-4ubuntu2.1
... python-dbg will load different (debug, _d) versions of the .so modules of libsvn (or python-subversion); and that is because I have the python-subversion-dbg package installed.
In any case, we may think we know the adresses where modules and respective functions are loaded upon each Python script call - which would allow us to place a gdb breakpoint on a program address; given that here we work with "vanilla" .so's (that haven't been rebuilt from source). However, Python on its own cannot see that _client.so in fact utilizes libsvn_client-1.so:
$ ls -la $(locate '*2.7*/_client*.so') #check locations
$ ls -la $(locate 'libsvn_client') #check locations
$ ldd /usr/lib/pyshared/python2.7/libsvn/_client.so | grep client
libsvn_client-1.so.1 => /usr/lib/libsvn_client-1.so.1 (0x0037f000)
#
# instead of nm, also can use:
# objdump -dSlr file | grep '^[[:digit:]].*status4' | grep -v '^$\|^[[:space:]]'
#
$ nm -D /usr/lib/pyshared/python2.7/libsvn/_client.so | grep status4
U svn_client_status4
$ nm -a /usr/lib/pyshared/python2.7/libsvn/_client_d.so | grep status4
00029a50 t _wrap_svn_client_status4
U svn_client_status4
$ nm -D /usr/lib/libsvn_client-1.so.1 | grep status4 # -a: no symbols
00038c10 T svn_client_status4
From within Python, we could make a system call, to query /proc/pid/maps for the address where libsvn_client-1.so is loaded, and add to it the address reported by the last nm -D command for the offset of svn_client_status4; and obtain the address where we could break in gdb (with the b *0xAddress syntax) - but that is not necessarry, because if nm can see the symbol, so can gdb - so we can break directly on the function name. Another thing is that in case of a segfault, gdb stops on its own, and we can issue a backtrace (note: use Ctrl-X A to exit the gdb TUI mode after layout asm):
$ gdb --args python python-subversion-test.py ./AudioFPGA/
(gdb) r
Starting program: /usr/bin/python python-subversion-test.py ./MyRepoWCDir
...
Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) bt
#0 0x00000000 in ?? ()
#1 0x005a5bf3 in ?? () from /usr/lib/libsvn_client-1.so.1
#2 0x005dbf4a in ?? () from /usr/lib/libsvn_wc-1.so.1
#3 0x005dcea3 in ?? () from /usr/lib/libsvn_wc-1.so.1
#4 0x005dd240 in ?? () from /usr/lib/libsvn_wc-1.so.1
#5 0x005a5fe5 in svn_client_status4 () from /usr/lib/libsvn_client-1.so.1
#6 0x00d54dae in ?? () from /usr/lib/pymodules/python2.7/libsvn/_client.so
#7 0x080e0155 in PyEval_EvalFrameEx ()
...
(gdb) frame 1
#1 0x005a5bf3 in ?? () from /usr/lib/libsvn_client-1.so.1
(gdb) list
No symbol table is loaded. Use the "file" command.
(gdb) disas
No function contains program counter for selected frame.
(gdb) x/10i 0x005a5bf3
=> 0x5a5bf3: mov -0xc(%ebp),%ebx
0x5a5bf6: mov -0x8(%ebp),%esi
0x5a5bf9: mov -0x4(%ebp),%edi
0x5a5bfc: mov %ebp,%esp
(gdb) layout asm # No function contains program counter for selected frame (cannot show 0x5a5bf3)
(gdb) p svn_client_status4
$1 = {<text variable, no debug info>} 0x5a5c10 <svn_client_status4>
(gdb) frame 5
#5 0x005a5fe5 in svn_client_status4 () from /usr/lib/libsvn_client-1.so.1
(gdb) list
No symbol table is loaded. Use the "file" command.
(gdb) layout asm
?0x5a5fd8 <svn_client_status4+968> mov %esi,0x4(%esp) |
?0x5a5fdc <svn_client_status4+972> mov %eax,(%esp) |
?0x5a5fdf <svn_client_status4+975> mov -0x28(%ebp),%eax |
?0x5a5fe2 <svn_client_status4+978> call *0x38(%eax) |
>?0x5a5fe5 <svn_client_status4+981> test %eax,%eax |
?0x5a5fe7 <svn_client_status4+983> jne 0x5a5ce3 <svn_client_status4+211> |
?0x5a5fed <svn_client_status4+989> jmp 0x5a5ee3 <svn_client_status4+723> |
?0x5a5ff2 <svn_client_status4+994> lea -0x1fac(%ebx),%eax |
?0x5a5ff8 <svn_client_status4+1000> mov %eax,(%esp) |
So, our error happens somewhere in libsvn_client-1.so, but in memory area before svn_client_status4 function start; and since we don't have debugging symbols - we cannot say much else than that. Using python-dbg may give bit different results:
Program received signal SIGSEGV, Segmentation fault.
0x005aebf0 in ?? () from /usr/lib/libsvn_client-1.so.1
(gdb) bt
#0 0x005aebf0 in ?? () from /usr/lib/libsvn_client-1.so.1
#1 0x005e4f4a in ?? () from /usr/lib/libsvn_wc-1.so.1
#2 0x005e5ea3 in ?? () from /usr/lib/libsvn_wc-1.so.1
#3 0x005e6240 in ?? () from /usr/lib/libsvn_wc-1.so.1
#4 0x005aefe5 in svn_client_status4 () from /usr/lib/libsvn_client-1.so.1
#5 0x00d61e9e in _wrap_svn_client_status4 (self=0x0, args=0x8471214)
at /build/buildd/subversion-1.6.12dfsg/subversion/bindings/swig/python/svn_client.c:10001
...
(gdb) frame 4
#4 0x005aefe5 in svn_client_status4 () from /usr/lib/libsvn_client-1.so.1
(gdb) list
9876 in /build/buildd/subversion-1.6.12dfsg/subversion/bindings/swig/python/svn_client.c
(gdb) p svn_client_status4
$1 = {<text variable, no debug info>} 0x5aec10 <svn_client_status4>
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
...
0x00497a20 0x004c8be8 Yes /usr/lib/pymodules/python2.7/libsvn/_core_d.so
0x004e9fe0 0x004f52c8 Yes /usr/lib/libsvn_swig_py2.7_d-1.so.1
0x004f9750 0x00501678 Yes (*) /usr/lib/libsvn_diff-1.so.1
0x0050f3e0 0x00539d08 Yes (*) /usr/lib/libsvn_subr-1.so.1
0x00552200 0x00572658 Yes (*) /usr/lib/libapr-1.so.0
0x0057ddb0 0x005b14b8 Yes (*) /usr/lib/libsvn_client-1.so.1
...
0x00c2a8f0 0x00d11cc8 Yes (*) /usr/lib/libxml2.so.2
0x00d3f860 0x00d6dc08 Yes /usr/lib/pymodules/python2.7/libsvn/_client_d.so
...
(*): Shared library is missing debugging information.
... but the list command still gives us a source line belonging to frame 5 (not frame 4), and we still don't know more about svn_client_status4: while the python-subversion modules are loaded in their debug versions, debugging information is missing for libsvn_client-1.so. So, time to rebuild from source.
It is the actual subversion that we need to rebuild, or rather it's library part - since we already have debug modules from python-subversion; the package on my system is called libsvn1:
$ apt-show-versions -r 'libsvn'
libsvn1/natty uptodate 1.6.12dfsg-4ubuntu2.1
$ apt-cache search 'libsvn' | grep 'dbg'
python-subversion-dbg - Python bindings for Subversion (debug extension)
... and there is no debug package for it. To rebuild from source, I went through apt-get source libsvn1, with dependencies manually found via apt-rdepends --build-depends --follow=DEPENDS subversion. There are more details in the full log - but here we can note that the source package can built both the SWIG Python bindings (that is, python-subversion) and the Subversion library (libsvn1). Also, I ran make install with a location out of the main kernel tree; that means, that one had to explicitly specify the source-built modules via LD environment variables:
$ ELD=/path/to/src/subversion-1.6.12dfsg/tmpinst/usr/local/lib
$ LD_LIBRARY_PATH=$ELD:$ELD/svn-python/libsvn LD_PRELOAD="$ELD/libsvn_client-1.so $ELD/svn-python/libsvn/_core.so" gdb --args python python-subversion-test.py ./MyRepoWCDir
One tricky thing here is that building SWIG debug modules requires a call with python-dbg; apparently just doing ./configure --enable-debug doesn't do that; and so, just _core.so, etc are produced, albeit with debugging information. If we then try to enforce its loading as with the above command, but with python-dbg, we will get undefined symbol: Py_InitModule4, because:
$ objdump -d $(which python) | grep '^\w.*InitMod'
0813b770 <Py_InitModule4>:
$ objdump -d $(which python-dbg) | grep '^\w.*InitMod'
08124740 <Py_InitModule4TraceRefs>:
... python-dbg has a different Py_InitModule4 function. That, however, wasn't a problem, because simply python was used (as in the above invocation), and gdb still allowed stepping through the relevant functions in the newly built libsvn (the mentioned Bash script gdb_py_so_test.sh, as an example builds a basic Swig module in both debug and release versions to confirm the right procedure).
With debugging symbols for libsvn, the function call stack looks like this (pasted a bit differently):
#5 0x0016e654 in svn_client_status4 (..., libsvn_client/status.c:369
#4 0x007fd209 in close_edit (..., libsvn_wc/status.c:2144
#3 0x007fafaa in get_dir_status (..., libsvn_wc/status.c:1033
#2 0x007fa4e7 in send_unversioned_item (..., libsvn_wc/status.c:722
#1 0x0016dd17 in tweak_status (..., libsvn_client/status.c:81
#0 0x00000000 in ?? ()
... and since the same library functions are also used by command line svn client, we can compare, in say, frame 5:
# `svn status`:
(gdb) p *(sb->real_status_func)
$3 = {svn_error_t *(void *, const char *, svn_wc_status2_t *, apr_pool_t *)} 0x805e199 <print_status>
...
# `python python-subversion-test.py`
(gdb) p *(svn_wc_status_func3_t*)sb->real_status_func
Cannot access memory at address 0x0
So, in case of a Python call to status4, sb->real_status_func is NULL, causing a segfault. The reason for this can be revealed once we start reading the source: in ./subversion/libsvn_client/deprecated.c, the definition for status3 has:
svn_client_status3(svn_revnum_t *result_rev,
const char *path,
const svn_opt_revision_t *revision,
svn_wc_status_func2_t status_func,
void *status_baton,
....
struct status3_wrapper_baton swb = { 0 };
swb.old_func = status_func;
swb.old_baton = status_baton;
return svn_client_status4(result_rev, path, revision, status3_wrapper_func,
&swb, depth, get_all, update, no_ignore,
ignore_externals, changelists, ctx, pool);
... that is, when status3 is called with a callback function, it creates a struct, and assigns the function to one of the struct properties - and then uses the struct in the further call to status4! Since status3 actually works from Python - the conclusion is that we cannot correctly call status4 from Python (since that would involve creating a C struct in Python); and that doesn't matter anyways, because we can call status3 from Python - which then itself calls status4!
Then why is status4 addressible from Python? Probably because swig simply autogenerated an interface for it... In any case, here is an example, where a trip to the debugger reveals the source of the problem - but not really a bug :) Solution? Don't use status4.
Going back to the UTF-8 failure, which occured with status2 and status3 - it was easier, given that now source built versions of the modules were available. The problem was obvious in the function entry_name_to_utf8, and by exploring it's argument name, one could first realize that the file name causing the problem, did indeed contain non-ascii - but still legal UTF-8 characters (see Program to check/look up UTF-8/Unicode characters in string on command line? - Super User). I have then used this .gdbinit, to make a Python class breakpoint for gdb, that would print out the filenames, and break only on match with the problematic one.
Then the question is - how come, the command line client svn status does not crash on the same filename? By stepping through both svn status and python python-subversion-test.py, one can compare the respective function call stacks:
# call stack Python module:
#
_wrap_svn_client_status3 subversion/bindings/swig/python/svn_client.c * allocs:
(svn_swig_py_get_pool_arg(args, SWIGTYPE_p_apr_pool_t, &_global_py_pool, &_global_pool))
svn_client_status3 subversion/libsvn_client/deprecated.c
svn_client_status4 subversion/libsvn_client/status.c
close_edit subversion/libsvn_wc/status.c
get_dir_status subversion/libsvn_wc/status.c
# call stack svn client:
#
main subversion/svn/main.c
svn_cl__status subversion/svn/status-cmd.c * allocs
(subpool = svn_pool_create(pool))
svn_client_status4 subversion/libsvn_client/status.c
close_edit subversion/libsvn_delta/cancel.c
close_edit subversion/libsvn_wc/status.c
get_dir_status subversion/libsvn_wc/status.c
# svn call stack:
# ... svn_client_status4 - starts pool
#
get_dir_status subversion/libsvn_wc/status.c
handle_dir_entry subversion/libsvn_wc/status.c
get_dir_status subversion/libsvn_wc/status.c
svn_io_get_dirents2 subversion/libsvn_subr/io.c
entry_name_to_utf8 subversion/libsvn_subr/io.c
svn_path_cstring_to_utf8 subversion/libsvn_subr/path.c
svn_utf_cstring_to_utf8 subversion/libsvn_subr/utf.c * from here, bad node->handle
convert_cstring subversion/libsvn_subr/utf.c
convert_to_stringbuf subversion/libsvn_subr/utf.c * here, bad node => fail
At this point, one encounters the fact that Subversion uses libapr (Apache Portable Runtime) for memory allocation; and it is in fact this part causing the failure - principally, the function apr_xlate_conv_buffer behaves differently in the two cases.
But, it can be rather difficult to see what the actual problem is here, because apr_xlate_conv_buffer uses an encoding in node->frompage, which is set to the define APR_LOCALE_CHARSET 1 - and that doesn't change between svn status and Python cases. To come down to this, I've copy-pasted everything related to APR string copying and allocation down the call stack, and reconstructed a simple example that builds a Swig module, that should just copy a string using APR runtime; that example is in the directory aprtest, built with the bash script build-aprtest.sh.
Thanks to that example, it was revealed that the UTF failure problem can be fixed by calling setlocale in C before any APR string memory allocation - for more about that test, see #15977257 - Using utf-8 input for cmd Python module. Correspondingly, all we need to do from Python is execute:
import locale
locale.setlocale(locale.LC_ALL, '')
... before any calls to svn.client (and thus to libsvn, and thus to libapr). And here we have yet another example, for a trip to the debugger, without really having a bug :)