我想使用Tor对网页发出多个GET请求.我想为每个请求使用不同的ipaddress.
import socks
import socket
socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", 9150)
socket.socket = socks.socksocket
import requests
print (requests.get('http://icanhazip.com')).content
Run Code Online (Sandbox Code Playgroud)
使用这个,我提出了一个请求.如何更改ipaddress以制作另一个?
我将 Scrapy 与 Privoxy 和 Tor 一起使用。这是我之前的问题Scrapy with Privoxy and Tor: how to renew IP,这是蜘蛛:
\n\nfrom scrapy.contrib.spiders import CrawlSpider\nfrom scrapy.selector import Selector\nfrom scrapy.http import Request\n\nclass YourCrawler(CrawlSpider):\n name = "****"\n start_urls = [\n \'https://****.com/listviews/titles.php\',\n ]\n allowed_domains = ["****.com"]\n\n def parse(self, response):\n # go to the urls in the list\n s = Selector(response)\n page_list_urls = s.xpath(\'///*[@id="tab7"]/article/header/h2/a/@href\').extract()\n for url in page_list_urls:\n yield Request(response.urljoin(url), callback=self.parse_following_urls, dont_filter=True)\n\n # Return back and go to bext page in div#paginat ul li.next a::attr(href) and begin …Run Code Online (Sandbox Code Playgroud)