在Python中使用代理运行Selenium Webdriver

use*_*813 59 python proxy selenium selenium-ide selenium-webdriver

我试图在Python中运行Selenium Webdriver脚本来执行一些基本任务.当通过Selenium IDE界面运行时,我可以让机器人完美地运行(即:当简单地让GUI重复我的动作时).但是,当我将代码导出为Python脚本并尝试从命令行执行时,Firefox浏览器将打开但无法访问起始URL(错误将返回到命令行并且程序停止).无论我试图访问什么网站等,这都发生在我身上.

我在这里包含了一个非常基本的代码用于演示目的.我不认为我已正确包含代码的代理部分,因为返回的错误似乎是由代理生成的.

任何帮助将非常感激.

以下代码仅用于打开www.google.ie并搜索"selenium"一词.对我来说它打开一个空白的Firefox浏览器并停止.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
import unittest, time, re
from selenium.webdriver.common.proxy import *

class Testrobot2(unittest.TestCase):
    def setUp(self):

        myProxy = "http://149.215.113.110:70"

        proxy = Proxy({
        'proxyType': ProxyType.MANUAL,
        'httpProxy': myProxy,
        'ftpProxy': myProxy,
        'sslProxy': myProxy,
        'noProxy':''})

        self.driver = webdriver.Firefox(proxy=proxy)
        self.driver.implicitly_wait(30)
        self.base_url = "https://www.google.ie/"
        self.verificationErrors = []
        self.accept_next_alert = True

    def test_robot2(self):
        driver = self.driver
        driver.get(self.base_url + "/#gs_rn=17&gs_ri=psy-ab&suggest=p&cp=6&gs_id=ix&xhr=t&q=selenium&es_nrs=true&pf=p&output=search&sclient=psy-ab&oq=seleni&gs_l=&pbx=1&bav=on.2,or.r_qf.&bvm=bv.47883778,d.ZGU&fp=7c0d9024de9ac6ab&biw=592&bih=665")
        driver.find_element_by_id("gbqfq").clear()
        driver.find_element_by_id("gbqfq").send_keys("selenium")

    def is_element_present(self, how, what):
        try: self.driver.find_element(by=how, value=what)
        except NoSuchElementException, e: return False
        return True

    def is_alert_present(self):
        try: self.driver.switch_to_alert()
        except NoAlertPresentException, e: return False
        return True

    def close_alert_and_get_its_text(self):
        try:
            alert = self.driver.switch_to_alert()
            alert_text = alert.text
            if self.accept_next_alert:
                alert.accept()
            else:
                alert.dismiss()
            return alert_text
        finally: self.accept_next_alert = True

    def tearDown(self):
        self.driver.quit()
        self.assertEqual([], self.verificationErrors)

if __name__ == "__main__":
    unittest.main()
Run Code Online (Sandbox Code Playgroud)

Ame*_*mey 26

这样的事情怎么样?

PROXY = "149.215.113.110:70"

webdriver.DesiredCapabilities.FIREFOX['proxy'] = {
    "httpProxy":PROXY,
    "ftpProxy":PROXY,
    "sslProxy":PROXY,
    "noProxy":None,
    "proxyType":"MANUAL",
    "class":"org.openqa.selenium.Proxy",
    "autodetect":False
}

# you have to use remote, otherwise you'll have to code it yourself in python to 
driver = webdriver.Remote("http://localhost:4444/wd/hub", webdriver.DesiredCapabilities.FIREFOX)
Run Code Online (Sandbox Code Playgroud)

你可以在这里阅读更多相关信息.


小智 24

以这种方式为我工作(类似于@Amey和@ user4642224代码,但更短一点):

from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy, ProxyType

prox = Proxy()
prox.proxy_type = ProxyType.MANUAL
prox.http_proxy = "ip_addr:port"
prox.socks_proxy = "ip_addr:port"
prox.ssl_proxy = "ip_addr:port"

capabilities = webdriver.DesiredCapabilities.CHROME
prox.add_to_capabilities(capabilities)

driver = webdriver.Chrome(desired_capabilities=capabilities)
Run Code Online (Sandbox Code Playgroud)

  • 这个答案对我不起作用,我收到错误“指定‘socksProxy’需要‘socksVersion’的整数”。 (11认同)
  • @Alex根据您使用的代理,只需添加“prox.socks_version = 5”或“prox.socks_version = 4”。这应该可以清除错误。 (3认同)
  • 这有效,谢谢。奇怪的是,文档说您需要使用远程驱动程序。 (2认同)
  • 多谢。这对我来说绝对有效。我删除了 prox.socks_proxy = "ip_addr:port" prox.ssl_proxy = "ip_addr:port" 并添加了 prox.https_proxy = "ip_addr:port" (2认同)

mrk*_*rki 14

我的解决方案

def my_proxy(PROXY_HOST,PROXY_PORT):
        fp = webdriver.FirefoxProfile()
        # Direct = 0, Manual = 1, PAC = 2, AUTODETECT = 4, SYSTEM = 5
        print PROXY_PORT
        print PROXY_HOST
        fp.set_preference("network.proxy.type", 1)
        fp.set_preference("network.proxy.http",PROXY_HOST)
        fp.set_preference("network.proxy.http_port",int(PROXY_PORT))
        fp.set_preference("general.useragent.override","whater_useragent")
        fp.update_preferences()
        return webdriver.Firefox(firefox_profile=fp)
Run Code Online (Sandbox Code Playgroud)

然后调用你的代码:

my_proxy(PROXY_HOST,PROXY_PORT)
Run Code Online (Sandbox Code Playgroud)

我有这个代码的问题,因为我传递一个字符串作为端口#:

 PROXY_PORT="31280"
Run Code Online (Sandbox Code Playgroud)

这个很重要:

int("31280")
Run Code Online (Sandbox Code Playgroud)

您必须传递整数而不是字符串,否则您的firefox配置文件将不会设置为正确的端口,并且通过代理连接将无法正常工作.


小智 13

这是一篇相当老的帖子,但是,对于其他人来说,它仍然可能通过提供今天的答案而受益,而且最初的作者非常接近一个可行的解决方案。

首先,此时不再支持ftpProxy设置,会抛出错误

proxy = Proxy({
        'proxyType': ProxyType.MANUAL,
        'httpProxy': myProxy,
        'ftpProxy': myProxy, # this will throw an error
        'sslProxy': myProxy,
        'noProxy':''})
Run Code Online (Sandbox Code Playgroud)

接下来,您应该使用 firefox 选项,而不是设置代理属性,如下所示

proxy = Proxy({
    'proxyType': ProxyType.MANUAL,
    'httpProxy': myProxy,
    'sslProxy': myProxy,
    'noProxy': ''})

options = Options()
options.proxy = proxy
driver = webdriver.Firefox(options=options)
Run Code Online (Sandbox Code Playgroud)

此外,在指定代理时不要定义方案,特别是如果您想对多个协议使用相同的代理

myProxy = "149.215.113.110:70"
Run Code Online (Sandbox Code Playgroud)

总的来说,它看起来像这样

from selenium import webdriver
from selenium.webdriver.common.proxy import *
from selenium.webdriver.firefox.options import Options

myProxy = "149.215.113.110:70"
proxy = Proxy({
    'proxyType': ProxyType.MANUAL,
    'httpProxy': myProxy,
    'sslProxy': myProxy,
    'noProxy': ''})

options = Options()
options.proxy = proxy
driver = webdriver.Firefox(options=options)
driver.get("https://www.google.ie")
Run Code Online (Sandbox Code Playgroud)


cho*_*ean 7

也尝试设置 sock5 代理。我遇到了同样的问题,它是通过使用袜子代理解决的

def install_proxy(PROXY_HOST,PROXY_PORT):
        fp = webdriver.FirefoxProfile()
        print PROXY_PORT
        print PROXY_HOST
        fp.set_preference("network.proxy.type", 1)
        fp.set_preference("network.proxy.http",PROXY_HOST)
        fp.set_preference("network.proxy.http_port",int(PROXY_PORT))
        fp.set_preference("network.proxy.https",PROXY_HOST)
        fp.set_preference("network.proxy.https_port",int(PROXY_PORT))
        fp.set_preference("network.proxy.ssl",PROXY_HOST)
        fp.set_preference("network.proxy.ssl_port",int(PROXY_PORT))  
        fp.set_preference("network.proxy.ftp",PROXY_HOST)
        fp.set_preference("network.proxy.ftp_port",int(PROXY_PORT))   
        fp.set_preference("network.proxy.socks",PROXY_HOST)
        fp.set_preference("network.proxy.socks_port",int(PROXY_PORT))   
        fp.set_preference("general.useragent.override","Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.75.14 (KHTML, like Gecko) Version/7.0.3 Safari/7046A194A")
        fp.update_preferences()
        return webdriver.Firefox(firefox_profile=fp)
Run Code Online (Sandbox Code Playgroud)

然后install_proxy ( ip , port ) 从您的程序中调用 。


小智 7

带验证的代理。这是一个全新的 Python 脚本,参考了 Mykhail Martsyniuk 示例脚本。

# Load webdriver
from selenium import webdriver

# Load proxy option
from selenium.webdriver.common.proxy import Proxy, ProxyType

# Configure Proxy Option
prox = Proxy()
prox.proxy_type = ProxyType.MANUAL

# Proxy IP & Port
prox.http_proxy = “0.0.0.0:00000”
prox.socks_proxy = “0.0.0.0:00000”
prox.ssl_proxy = “0.0.0.0:00000”

# Configure capabilities 
capabilities = webdriver.DesiredCapabilities.CHROME
prox.add_to_capabilities(capabilities)

# Configure ChromeOptions
driver = webdriver.Chrome(executable_path='/usr/local/share chromedriver',desired_capabilities=capabilities)

# Verify proxy ip
driver.get("http://www.whatsmyip.org/")
Run Code Online (Sandbox Code Playgroud)


小智 6

如果有人在这里寻找解决方案的方法如下:

from selenium import webdriver
PROXY = "YOUR_PROXY_ADDRESS_HERE"
webdriver.DesiredCapabilities.FIREFOX['proxy']={
    "httpProxy":PROXY,
    "ftpProxy":PROXY,
    "sslProxy":PROXY,
    "noProxy":None,
    "proxyType":"MANUAL",
    "autodetect":False
}
driver = webdriver.Firefox()
driver.get('http://www.whatsmyip.org/')
Run Code Online (Sandbox Code Playgroud)


Dug*_*jay 6

上述结果可能是正确的,但不适用于最新的网络驱动程序。这是我对上述问题的解决方案。简单又甜蜜


        http_proxy  = "ip_addr:port"
        https_proxy = "ip_addr:port"

        webdriver.DesiredCapabilities.FIREFOX['proxy']={
            "httpProxy":http_proxy,
            "sslProxy":https_proxy,
            "proxyType":"MANUAL"
        }

        driver = webdriver.Firefox()
Run Code Online (Sandbox Code Playgroud)

或者

    http_proxy  = "http://ip:port"
    https_proxy = "https://ip:port"

    proxyDict = {
                    "http"  : http_proxy,
                    "https" : https_proxy,
                }

    driver = webdriver.Firefox(proxy=proxyDict)
Run Code Online (Sandbox Code Playgroud)


Jac*_*ssn 5

这对我在 2022 年 9 月有帮助 - 具有身份验证用户+密码的 selenium 代理

import os
import zipfile

from selenium import webdriver

PROXY_HOST = '192.168.3.2'  # rotating proxy or host
PROXY_PORT = 8080 # port
PROXY_USER = 'proxy-user' # username
PROXY_PASS = 'proxy-password' # password

manifest_json = """
{
    "version": "1.0.0",
    "manifest_version": 2,
    "name": "Chrome Proxy",
    "permissions": [
        "proxy",
        "tabs",
        "unlimitedStorage",
        "storage",
        "<all_urls>",
        "webRequest",
        "webRequestBlocking"
    ],
    "background": {
        "scripts": ["background.js"]
    },
    "minimum_chrome_version":"22.0.0"
}
"""

background_js = """
var config = {
        mode: "fixed_servers",
        rules: {
        singleProxy: {
            scheme: "http",
            host: "%s",
            port: parseInt(%s)
        },
        bypassList: ["localhost"]
        }
    };
chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});
function callbackFn(details) {
    return {
        authCredentials: {
            username: "%s",
            password: "%s"
        }
    };
}
chrome.webRequest.onAuthRequired.addListener(
            callbackFn,
            {urls: ["<all_urls>"]},
            ['blocking']
);
""" % (PROXY_HOST, PROXY_PORT, PROXY_USER, PROXY_PASS)

def get_chromedriver(use_proxy=False, user_agent=None):
    path = os.path.dirname(os.path.abspath(__file__))
    chrome_options = webdriver.ChromeOptions()
    if use_proxy:
        pluginfile = 'proxy_auth_plugin.zip'

        with zipfile.ZipFile(pluginfile, 'w') as zp:
            zp.writestr("manifest.json", manifest_json)
            zp.writestr("background.js", background_js)
        chrome_options.add_extension(pluginfile)
    if user_agent:
        chrome_options.add_argument('--user-agent=%s' % user_agent)
    driver = webdriver.Chrome(
        os.path.join(path, 'chromedriver'),
        chrome_options=chrome_options)
    return driver

def main():
    driver = get_chromedriver(use_proxy=True)
    driver.get('https://ifconfig.me/)

if __name__ == '__main__':
    main()
Run Code Online (Sandbox Code Playgroud)

来源链接