如何发出模拟浏览器的Web请求（即TLS握手/客户端问候）？

Question

我想知道如何发出一个典型的网络请求（例如使用 python 请求），它将能够完美地模拟网络浏览器。我不是在谈论微不足道的用户代理，而是在谈论 Client Hello 消息，其中包括密码套件顺序等内容。

例如，我向https://www.ssllabs.com/ssltest/viewMyClient.html发出请求，保存输出并将结果与​​常规 Chrome 浏览器显示的结果进行比较。

这是我到目前为止所拥有的（非常丑陋......我正在使用多个适配器）

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.ssl_ import create_urllib3_context
import re
import json
import random
import subprocess
import os



from requests.packages.urllib3.poolmanager import PoolManager
from requests.packages.urllib3.util import ssl_
from urllib3.poolmanager import PoolManager

from requests.adapters import HTTPAdapter



from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.ssl_ import create_urllib3_context



# This is the 2.11 Requests cipher string, containing 3DES.
CIPHERS = (
    'ECDH+AESGCM:ECDH+AES256:ECDH+AES128:'
    'RSA+AESGCM'
)


class DESAdapter(HTTPAdapter):
    """
    A TransportAdapter that re-enables 3DES support in Requests.
    """
    def init_poolmanager(self, *args, **kwargs):
        context = create_urllib3_context(ciphers=CIPHERS)
        context.options |= ssl.OP_NO_SSLv3
        context.options |= ssl.OP_NO_SSLv2
        context.options |= ssl.OP_NO_SSLv3
        context.options |= ssl.OP_NO_TLSv1
        context.options |= ssl.OP_NO_TLSv1_1
        kwargs['ssl_context'] = context

        return super(DESAdapter, self).init_poolmanager(*args, **kwargs)

    def proxy_manager_for(self, *args, **kwargs):
        context = create_urllib3_context(ciphers=CIPHERS)
        context.options |= ssl.OP_NO_SSLv3
        kwargs['ssl_context'] = context

        return super(DESAdapter, self).proxy_manager_for(*args, **kwargs)




class Ssl3HttpAdapter(HTTPAdapter):
    """"Transport adapter" that allows us to use SSLv3."""

    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(
            num_pools=connections, maxsize=maxsize,
            block=block, ssl_version=ssl.OP_NO_SSLv3)




CIPHERS = (
    'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:'
    'ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:AES128-GCM-SHA256:'
    'AES256-GCM-SHA384:AES128-SHA:AES256-SHA:3DES'   #TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
    )



class TlsAdapter(HTTPAdapter):

    def __init__(self, ssl_options=0, **kwargs):
        self.ssl_options = ssl_options
        super(TlsAdapter, self).__init__(**kwargs)

    def init_poolmanager(self, *pool_args, **pool_kwargs):
        ctx = ssl_.create_urllib3_context(ciphers=CIPHERS,options=self.ssl_options)
        self.poolmanager = PoolManager(*pool_args,
                                       ssl_context=ctx,
                                       **pool_kwargs)


s = requests.session()
adapter = TlsAdapter(ssl.OP_NO_SSLv3)
s.mount("https://", adapter)
r = s.get('https://www.ssllabs.com/ssltest/viewMyClient.html')



if 'Capabilities' in r.text:   
  os.remove("ssl-test-python.html")
  hs = open("ssl-test-python.html", "a")
  hs.write(r.text)
  hs.close()

  os.startfile('ssl-test-python.html')

将保存的输出与 chrome 显示的内容进行比较时，主要的不一致似乎是：

1）密码的顺序不正确

2）不应启用 sslv3

3）显示了太多的“签名算法”

4）“命名”组不正确

5) 可能还有其他东西我丢失了

有什么办法可以解决这些问题吗？我不想使用硒

Answer 1

现在你可以用python完美模拟浏览器了！使用curl_cffi。它是curl-impersonate 项目的Python 绑定。

pip install curl_cffi

pip install curl_cffi