isinstance()意外返回False

Question

我正在使用kubernetes python客户端.如果我的代码启动时kubernetes不可用,我想重试连接.

当客户端无法连接时,它会抛出看似urllib3.exceptions.MaxRetryError异常的内容,所以我开始使用以下内容:

import time
import urllib3

import kubernetes

kubernetes.config.load_kube_config()
api = kubernetes.client.CoreV1Api()

while True:
    try:
        w = kubernetes.watch.Watch()
        for event in w.stream(api.list_pod_for_all_namespaces):
            print event
    except urllib3.exceptions.HTTPError:
        print('retrying in 1 second')
        time.sleep(1)

但那完全失败了; 它的行为就像没有任何except声明一样,并且誓言:

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='192.168.122.140', port=8443): Max retries exceeded with url: /api/v1/pods?watch=True (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x2743110>: Failed to establish a new connection: [Errno 111] Connection refused',))

我想也许我不理解继承和我想的,所以我用以下内容替换上面的:

except urllib3.exceptions.MaxRetryError:
    print('retrying in 1 second')
    time.sleep(1)

哪个失败的方式相同.为了弄清楚发生了什么,我添加了一个catch-all except和invoked pdb:

except Exception as err:
    import pdb; pdb.set_trace()

从pdb提示中,我们可以看到:

(Pdb) type(err)
<class 'urllib3.exceptions.MaxRetryError'>

...看起来很好,和mro一样:

(Pdb) import inspect
(Pdb) inspect.getmro(err.__class__)
(<class 'urllib3.exceptions.MaxRetryError'>, <class 'urllib3.exceptions.RequestError'>, <class 'urllib3.exceptions.PoolError'>, <class 'urllib3.exceptions.HTTPError'>, <type 'exceptions.Exception'>, <type 'exceptions.BaseException'>, <type 'object'>)

但尽管如此:

(Pdb) isinstance(err, urllib3.exceptions.MaxRetryError)
False

并且所有路径看起来都合理:

(Pdb) urllib3.__file__
'/usr/lib/python2.7/site-packages/urllib3/__init__.pyc'
(Pdb) kubernetes.client.rest.urllib3.__file__
'/usr/lib/python2.7/site-packages/urllib3/__init__.pyc'

那么......实际发生了什么？

更新

这是完整的堆栈跟踪:

Traceback (most recent call last):
  File "testkube.py", line 13, in <module>
    for event in w.stream(api.list_pod_for_all_namespaces):
  File "/usr/lib/python2.7/site-packages/kubernetes/watch/watch.py", line 116, in stream
    resp = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 14368, in list_pod_for_all_namespaces
    (data) = self.list_pod_for_all_namespaces_with_http_info(**kwargs)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/apis/core_v1_api.py", line 14464, in list_pod_for_all_namespaces_with_http_info
    collection_formats=collection_formats)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 335, in call_api
    _preload_content, _request_timeout)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 148, in __call_api
    _request_timeout=_request_timeout)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/api_client.py", line 371, in request
    headers=headers)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/rest.py", line 250, in GET
    query_params=query_params)
  File "/usr/lib/python2.7/site-packages/kubernetes/client/rest.py", line 223, in request
    headers=headers)
  File "/usr/lib/python2.7/site-packages/urllib3/request.py", line 66, in request
    **urlopen_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/request.py", line 87, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/poolmanager.py", line 321, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/lib/python2.7/site-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='192.168.122.140', port=8443): Max retries exceeded with url: /api/v1/pods?watch=True (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x3d16110>: Failed to establish a new connection: [Errno 111] Connection refused',))

Answer 1

您的代码与requests包有间接依赖关系,并且requests包中有一个奇怪的子模块requests.packages.这用于包含来自许多依赖项的复制源代码,包括urllib3但是他们停止这样做.他们想要保持requests.packages向后兼容,所以现在他们做了一些奇怪的事情.

它现在不是requests.packages包含urllib3源代码的完整副本,而是导入urllib3和设置sys.modules['requests.packages.urllib3'] = urllib3.根据requests版本的不同,它也可以设置许多其他sys.modules条目; 例如,从请求2.18.4开始,源代码就是这样做的

for package in ('urllib3', 'idna', 'chardet'):
    locals()[package] = __import__(package)
    # This traversal is apparently necessary such that the identities are
    # preserved (requests.packages.urllib3.* is urllib3.*)
    for mod in list(sys.modules):
        if mod == package or mod.startswith(package + '.'):
            sys.modules['requests.packages.' + mod] = sys.modules[mod]

但在2.17.0中确实如此

import urllib3
sys.modules['requests.packages.urllib3'] = urllib3

import idna
sys.modules['requests.packages.idna'] = idna

import chardet
sys.modules['requests.packages.chardet'] = chardet

此代码与导入的包的子模块严重交互.如果你的代码试图做import requests.packages.urllib3.exceptions和Python没有找到sys.modules['requests.packages.urllib3.exceptions']入口,Python将重建的urllib3.exceptions模块,并设置urllib3.exceptions和sys.modules['requests.packages.urllib3.exceptions']新的模块(但不会碰sys.modules['urllib3.exceptions'],这会产生所涉及的类的新副本,造成您的错误.

5月份报告了一个相同原因的相关问题,导致2.18.4中显示的新代码.2.18.4应该不会造成您看到的具体问题,但它仍然是脆弱的,因为如果任何子模块urllib3尚未加载的时候requests.packages用螺钉sys.modules,这些子模块将展示你今天已经看到了同样的问题.