我试图单击以下类名 equals 的元素"clean right":
<li class="clean right"></li>
Run Code Online (Sandbox Code Playgroud)
我怎样才能通过使用找到它driver.find_element_by_class_name()
当尝试通过 ('url' : response.request.url) 在字典中保存 URL 时,Scrapy 会保存来自 Scrapy-Splash 的 URL,它们都是相同的(http://localhost:8050/render.html)
我试过添加额外的参数来传递真实的 URL 无济于事。
from scrapy import Spider
from scrapy.http import FormRequest
from scrapy.utils.response import open_in_browser
from scrapy import Request
import scrapy
from scrapy_splash import SplashRequest
class QuotesJSSpider(scrapy.Spider):
name = 'quotesjs'
start_urls = ('https://www.facebook.com/login',)
custom_settings = {
'SPLASH_URL': 'http://localhost:8050',
'DOWNLOADER_MIDDLEWARES': {
'scrapy_splash.SplashCookiesMiddleware': 723,
'scrapy_splash.SplashMiddleware': 725,
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': 810,
},
'SPIDER_MIDDLEWARES': {
'scrapy_splash.SplashDeduplicateArgsMiddleware': 100,
},
'DUPEFILTER_CLASS': 'scrapy_splash.SplashAwareDupeFilter',
}
def parse(self, response):
token = response.xpath('//*[@id="u_0_a"]').extract_first()
return FormRequest.from_response(response,
formdata={'lgndim' : token,
'pass': …Run Code Online (Sandbox Code Playgroud)