相关疑难解决方法(0)

使用Python over Tor发出请求

我想使用Tor对网页发出多个GET请求.我想为每个请求使用不同的ipaddress.

import socks
import socket
socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", 9150)
socket.socket = socks.socksocket
import requests
print (requests.get('http://icanhazip.com')).content
Run Code Online (Sandbox Code Playgroud)

使用这个,我提出了一个请求.如何更改ipaddress以制作另一个?

python tor

51
推荐指数
6
解决办法
6万
查看次数

Scrapy、privoxy 和 Tor:SocketError:[Errno 61] 连接被拒绝

我将 Scrapy 与 Privoxy 和 Tor 一起使用。这是我之前的问题Scrapy with Privoxy and Tor: how to renew IP,这是蜘蛛:

\n\n
from scrapy.contrib.spiders import CrawlSpider\nfrom scrapy.selector import Selector\nfrom scrapy.http import Request\n\nclass YourCrawler(CrawlSpider):\n    name = "****"\n    start_urls = [\n    \'https://****.com/listviews/titles.php\',\n    ]\n    allowed_domains = ["****.com"]\n\n    def parse(self, response):\n        # go to the urls in the list\n        s = Selector(response)\n        page_list_urls = s.xpath(\'///*[@id="tab7"]/article/header/h2/a/@href\').extract()\n        for url in page_list_urls:\n            yield Request(response.urljoin(url), callback=self.parse_following_urls, dont_filter=True)\n\n        # Return back and go to bext page in div#paginat ul li.next a::attr(href) and begin …
Run Code Online (Sandbox Code Playgroud)

python tor scrapy web-scraping

5
推荐指数
1
解决办法
3232
查看次数

标签 统计

python ×2

tor ×2

scrapy ×1

web-scraping ×1