标签: urllib3

我是Python Requests的新手,遇到了IOError:[Errno 22] Invalid argument我尝试的时候requests.get().简而言之,我正在尝试使用SSL连接到内部Web应用程序,因此我根据请求文档传递证书/密钥组合.

我花了相当多的时间研究潜在的问题,并看到一些潜在的SNI问题,但我不够精明,不知道如何解决问题(再次,新的请求).欣赏正确方向的任何推动/进一步深入挖掘(我猜urllib3件？)

我的守则

import requests

cert_file_path = "/Users/me/Documents/cert.pem"
key_file_path = "/Users/me/Documents/key.pem"

url = "https://mydomain/path/to/something"
cert = (cert_file_path, key_file_path)
r = requests.get(url, cert=cert) 

我的错误

IOError                               Traceback (most recent call last)
    <ipython-input-48-1ee4a7f23d00> in <module>()
          4 url = "https://mydomain/path/to/something"
          5 cert = (cert_file_path, key_file_path)
    ----> 6 r = requests.get(url, cert=cert)

    /Users/me/anaconda/lib/python2.7/site-packages/requests/api.pyc in get(url, **kwargs)
         66 
         67     kwargs.setdefault('allow_redirects', True)
    ---> 68     return request('get', url, **kwargs)
         69 
         70 

    /Users/me/anaconda/lib/python2.7/site-packages/requests/api.pyc in request(method, url, **kwargs)
         48 
         49     session = …

我使用以下代码在 python 请求中实现重试机制。

from requests import Session
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

s = Session()
retries = Retry(total=5,
                raise_on_redirect=True, 
                raise_on_status=True,
                backoff_factor=1,
                status_forcelist=[ 500, 502, 503, 504 ])
s.mount('http://', HTTPAdapter(max_retries=retries))
s.mount('https://', HTTPAdapter(max_retries=retries))
response=s.get('https://example.com')

它工作得很好，然而这一切都悄无声息地发生。我想要的是每当进行重试尝试时都得到通知，如果可能的话，为什么会发生重试。我希望每次重试都有这样的事情。

print(Exception.__class__)
print('Retrying after' + x + 'seconds')

是否可以？

当使用__del__ datetime.date.today() 抛出 ImportError: sys.meta_path is None 时，Python 可能会关闭

import datetime
import time
import sys


class Bug(object):

    def __init__(self):
        print_meta_path()

    def __del__(self):
        print_meta_path()
        try_date('time')
        try_date('datetime')


def print_meta_path():
    print(f'meta_path: {sys.meta_path}')


def try_date(date_type):
    try:
        print('----------------------------------------------')
        print(date_type)
        if date_type == 'time':
            print(datetime.date.fromtimestamp(time.time()))
        if date_type == 'datetime':
            print(datetime.date.today())
    except Exception as ex:
        print(ex)


if __name__ == '__main__':
    print(sys.version)
    bug = Bug()

不同环境的输出（3.10、3.9、3.7）：

3.10.4 | packaged by conda-forge | (main, Mar 24 2022, 17:39:04) [GCC 10.3.0]
meta_path: [<_distutils_hack.DistutilsMetaFinder object at 0x7ff8731f6860>, <class '_frozen_importlib.BuiltinImporter'>, <class …

我正在尝试诊断一个问题,我的一些芹菜工人进程似乎挂了几分钟.我有许多任务可以进行多次IO调用(通常是第三方API).在任何给定的工作中,我可能会对各种API提出数千个请求.我查看了日志,它们都有共同点:它们在urllib3连接到远程URL 后挂起.

在我的工作结束时(大约需要30分钟),通常会挂起一些任务.

以下是我用来推断的日志的例子urllib3:罪魁祸首:

Jul 08 04:46:26 app/worker.1:  [INFO/MainProcess] [???(???)] celery.worker.strategy: Received task: my_celery_task[734a49f6-bf6b-4423-9146-1c48366ba897] 
Jul 08 04:46:28 app/worker.1:  [DEBUG/Worker-11] [my_celery_task(734a49f6-bf6b-4423-9146-1c48366ba897)] src.aggregates.prospect.services.prospect_service: Beginning: Get social account data. provider_name: twitter, account_uid: some_user 
Jul 08 04:46:28 app/worker.1:  [INFO/Worker-11] [my_celery_task(734a49f6-bf6b-4423-9146-1c48366ba897)] requests.packages.urllib3.connectionpool: Starting new HTTPS connection (1): api.some_api.com 

然后就是这样.Starting new HTTPS connection声明后没有任何记录.

这是我重新启动worker的地方:

Jul 08 05:09:18 app/worker.1:  [INFO/MainProcess] [???(???)] celery.worker.strategy: Received task: my_celery_task[734a49f6-bf6b-4423-9146-1c48366ba897] 
Jul 08 05:09:19 app/worker.1:  [DEBUG/Worker-4] [my_celery_task(734a49f6-bf6b-4423-9146-1c48366ba897)] src.aggregates.prospect.services.prospect_service: Beginning: Get social account data. provider_name: twitter, account_uid: some_user …

我在构建Docker镜像时收到此警告:

/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:79: 
      InsecurePlatformWarning: A true SSLContext object is not available. 
      This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. 
      For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

几个来源(如InsecurePlatformWarning:一个真正的SSLContext对象不可用.这可以防止urllib3正确配置SSL)说这pip install pyopenssl ndg-httpsclient pyasn1将解决这个问题.但是一旦pip尝试安装pyopenssl,我就会收到警告.

这是我的Dockerfile:

FROM ubuntu:14.04

# Install packages
RUN apt-get update && apt-get install -y \
    git \
    libmysqlclient-dev \
    mysql-server \
    nginx \
    python-dev \
    python-mysqldb \
    python-setuptools \
    supervisor \
    vim
RUN easy_install pip

# Handle urllib3 InsecurePlatformWarning
RUN apt-get install …

操作系统:Mac OS X.当我尝试运行下面的代码时,我收到错误:

ImportError:无法导入名称HeaderParsingError

我在代码下面附加了回溯.

我已经尝试使用谷歌和其他stackoverflow解决这个问题20分钟了.我试过跑:

pip install urllib3 --upgrade

我也尝试重新安装请求包.

它没有帮助.

这似乎是我的请求或urllib3包的问题.有没有人有类似的问题？

代码:

import requests
import json


def printResponse(r):
print '{} {}\n'.format(json.dumps(r.json(), sort_keys=True, indent=4,    separators=(',', ': ')), r)


r = requests.get('http://wikitest.orcsoftware.com/rest/api/content',
             params={'title': 'new page'},
             auth=('seb', '****'))
printResponse(r)
parentPage = r.json()['results'][0]
pageData = {'type': 'comment', 'container': parentPage,
        'body': {'storage': {'value': "<p>A new comment</p>", 'representation': 'storage'}}}
r =    requests.post('http://localhost:8080/confluence/rest/api/content',
              data=json.dumps(pageData),
              auth=('admin', 'admin'),
              headers=({'Content-Type': 'application/json'}))
printResponse(r)

这是追溯:

Traceback (most recent call last):
  File "/Users/sebastian/OneDrive/orc/restAPI/createSpace.py", line 1, in <module>
    import requests
  File "/Library/Python/2.7/site-packages/requests/__init__.py", …

我已经搜索了这个,但我遇到的其他例子是人们在安装请求时遇到问题,我的问题是导入模块:

使用Putty(连接到HDF 2.4沙箱会话)和Python来调用API.

昨晚,我有这个工作 - 安装Requests模块没问题,定义了Python,然后发出了GET请求,工作得很完美.

今晚,我按照完全相同的步骤,但当我尝试'导入请求'时,我收到错误:

ImportError: No module named urllib3

我昨晚复制的步骤是:

[root@sandbox ~]# pip install requests  
python –v
import requests 

图我必须无意中改变一些事情......任何人都可以建议吗？

我试图在Kubuntu 14.04上运行python中的selenium.我尝试使用chromedriver或geckodriver时出现此错误消息,两者都是相同的错误.

Traceback (most recent call last):
  File "vse.py", line 15, in <module>
    driver = webdriver.Chrome(chrome_options=options, executable_path=r'/root/Desktop/chromedriver')
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/chrome/webdriver.py", line 75, in __init__
    desired_capabilities=desired_capabilities)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 156, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 251, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 318, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/remote_connection.py", line 375, in execute
    return self._request(command_info[0], url, body=data)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/remote_connection.py", line 397, in _request
    resp = self._conn.request(method, url, body=body, headers=headers)
  File "/usr/lib/python3/dist-packages/urllib3/request.py", line 79, in request …

该certifi库经常遇到域证书问题，但在标准urllib.request库中没有问题。

如果我将上下文设置为使用证书文件，则会收到 SSL 错误，

import ssl
import certifi
import requests.urllib as urlrq

resp = urlrq.urlopen(url="https://<web address>/rest/info?f=json", 
                                  context=ssl.create_default_context(cafile=certifi.where()))

如果我使用标准库并设置上下文如下：

import ssl
import certifi
import requests.urllib as urlrq

resp = urlrq.urlopen(url="https://<web address>/rest/info?f=json", 
                                  context=ssl.SSLContext(ssl.PROTOCOL_TLSv1))

没有 SSL 错误。

我怎样才能收到尊重这一点的请求SSLContext？

我有一个网络抓取脚本，它在我的（Windows）PC 上运行良好，但我试图让它从（Linux）网络服务器上运行。我有许多其他脚本在服务器上运行良好（连接到与此不同的网站），但是当我运行此脚本时，出现[Errno 111] Connection refused错误。

这是演示问题的脚本的最小版本：

import time
import requests
import urllib.request
from bs4 import BeautifulSoup

s = requests.Session()

target = "http://taxsearch.co.grayson.tx.us:8443/"
headers = {"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
           "Accept-Encoding": "gzip, deflate",
           "Accept-Language": "en",
           "Cache-Control": "no-cache",
           "Connection": "keep-alive",
           "Host": "taxsearch.co.grayson.tx.us:8443",
           "Pragma": "no-cache",
           "Upgrade-Insecure-Requests": "1",
           "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36"
}


time.sleep(1)
response = s.get(target, headers=headers)

if response.status_code == requests.codes.ok:
    results = BeautifulSoup(response.text, 'html.parser')

    # Do something with output
else:
    response.raise_for_status()

这在我的 PC 上运行良好，但在服务器上运行时，出现以下错误： …