SSL ConnectionResetError来自何处？

Question

TL; DR

其中负责的代码-我的问题很简单,提高 ConnectionResetError对cpython3以下调用self._sslobj.read(len, buffer)上ssl.py？

背景

我有时候ConnectionResetError尝试用ssl连接S3.这个错误很少发生,所以重现它很棘手.

# trimmed stacktrace
File "/MYPROJECT/MY_FUNC.py", line 123, in <genexpr>
rows = (row for row in reader)
File "/XXX/lib/python3.6/csv.py", line 112, in _next_
row = next(self.reader)
File "/XXX/lib/python3.6/tarfile.py", line 706, in readinto
buf = self.read(len(b))
File "/XXX/lib/python3.6/tarfile.py", line 695, in read
b = self.fileobj.read(length)
File "/XXX/lib/python3.6/gzip.py", line 276, in read
return self._buffer.read(size)
File "/XXX/lib/python3.6/_compression.py", line 68, in readinto
data = self.read(len(byte_view))
File "/XXX/lib/python3.6/gzip.py", line 469, in read
buf = self._fp.read(io.DEFAULT_BUFFER_SIZE)
File "/XXX/lib/python3.6/gzip.py", line 91, in read
self.file.read(size-self._length+read)
File "/XXX/lib/python3.6/site-packages/s3fs/core.py", line 1311, in read
self._fetch(self.loc, self.loc + length)
File "/XXX/lib/python3.6/site-packages/s3fs/core.py", line 1292, in _fetch
req_kw=self.s3.req_kw)
File "/XXX/lib/python3.6/site-packages/s3fs/core.py", line 1496, in _fetch_range
return resp['Body'].read()
File "/XXX/lib/python3.6/site-packages/botocore/response.py", line 74, in read
chunk = self._raw_stream.read(amt)
File "/XXX/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/response.py", line 239, in read
data = self._fp.read()
File "/XXX/lib/python3.6/http/client.py", line 462, in read
s = self._safe_read(self.length)
File "/XXX/lib/python3.6/http/client.py", line 612, in _safe_read
chunk = self.fp.read(min(amt, MAXAMOUNT))
File "/XXX/lib/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
File "/XXX/lib/python3.6/ssl.py", line 1009, in recv_into
return self.read(nbytes, buffer)
File "/XXX/lib/python3.6/ssl.py", line 871, in read
return self._sslobj.read(len, buffer)
File "/XXX/lib/python3.6/ssl.py", line 631, in read
v = self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer

我试过的

看着ssl.py:631我没有进一步的线索 - 我们必须更深入!:

    def read(self, len=1024, buffer=None):
    """Read up to 'len' bytes from the SSL object and return them.

    If 'buffer' is provided, read into this buffer and return the number of
    bytes read.
    """
    if buffer is not None:
        v = self._sslobj.read(len, buffer)  # <--- exception here
    else:
        v = self._sslobj.read(len)
    return v

我试过在CPython repo上搜索它,但是AFAICS似乎没有提出它,我怀疑它隐藏在SSL实现中或者在子类之间的某些映射OSError上ConnectionError.

我的最终目标是ConnectionError通过比较引发此错误的模块的py2和py3版本来编写py2和py3兼容代码来处理此异常(在py3上是新的).

更新 - ConnectionError子类的py2和py3 catch

我的问题来源是ConnectionError在python2和python3上找到捕获及其子类的方法,所以这里是:

import errno

# ref: https://docs.python.org/3/library/exceptions.html#ConnectionError
_CONNECTION_ERRORS = frozenset({
    errno.ECONNRESET,  # ConnectionResetError
    errno.EPIPE, errno.ESHUTDOWN,  # BrokenPipeError
    errno.ECONNABORTED,  # ConnectionAbortedError
    errno.ECONNREFUSED,  # ConnectionRefusedError
})

try:
    ...
except OSError as e:
    if e.errno not in _CONNECTION_ERRORS:
        raise
    print('got ConnectionError - %e' % e)

Answer 1

ConnectionResetError在时errno被提高ECONNRESET。errno是libc指示系统调用中是否发生错误的方式。

你可以搜索ConnectionResetError在Objects/exceptions.c找出如何异常类型得到初始化，并添加到errnomap字典。

如果用self._sslobj.read凸起的来实现ConnectionResetError，_sslobj.read则使用来实现_ssl__SSLSocket_read_impl实际的ssl读取，它是使用openssl的SSL_read：

count = SSL_read(self->ssl, mem, len);
_PySSL_UPDATE_ERRNO_IF(count <= 0, self, count);

当发生错误时，_PySSL_UPDATE_ERRNO_IF将设置(sock)->ssl_errno = SSL_ERROR_SYSCALL和(sock)->c_errno = ECONNRESET。

以后，在PySSL_SetError：

    err = obj->ssl_errno;
    switch (err) {
    ...
    case SSL_ERROR_SYSCALL:

        if (obj->c_errno) {
            errno = obj->c_errno;
            return PyErr_SetFromErrno(PyExc_OSError);
        }

PyErr_SetFromErrno(PyExc_OSError) 等于：

OSError(errno.ECONNRESET, 'Connection reset by peer', ...)

当OSError使用an构造时errno，它将通过errno上述errnomapdict 中的lookup 值查找一个更指定的子类：

newtype = PyDict_GetItem(errnomap, myerrno);
if (newtype) {
    assert(PyType_Check(newtype));
    type = (PyTypeObject *) newtype;
}

它实际上返回并引发ConnectionResetError异常。