我正在使用scrapy + selenium,因为我正在废弃的网站需要javascript进行身份验证.我使用selenium登录并将cookie传递给以下请求.
def login(self, response):
driver = webdriver.Firefox()
driver.get("http://www.site.com/login")
driver.find_element_by_xpath("//input[@id='myname']").send_keys(settings['USERNAME'])
driver.find_element_by_xpath("//input[@id='mypwd']").send_keys(settings['PASSWORD'])
driver.find_element_by_xpath("//input[@name='Logon']").click()
self.driver = driver
return Request(url=driver.current_url, cookies=self.driver.get_cookies(), callback=self.after_login, dont_filter=True)
Run Code Online (Sandbox Code Playgroud)
到目前为止一切都很好,因为cookie很粘,以下所有请求都能很好地工作.我的报废很长,所以在某些时候cookie过期,所以我需要重新登录.此时我正在传递一个带回调登录功能的新请求.这里失败了,因为新的cookie与旧的cookie合并.有没有办法重置cookie?
回答
@Drewness在他的回答中建议dont_merge_cookies在元字典中使用属性.由于以下原因,它无法正常工作.根据源代码,以下请求:
Request(url=driver.current_url, cookies=self.driver.get_cookies(), callback=self.after_login, meta={'dont_merge_cookies' : True}, dont_filter=True)
Run Code Online (Sandbox Code Playgroud)
你传给他的饼干什么都不做.
在我的解决方案中,我决定跳过dont_merge_cookies属性,只需在创建请求之前重置响应头:
response.headers = {}
return Request(url=driver.current_url, cookies=self.driver.get_cookies(), callback=self.after_login, dont_filter=True)
Run Code Online (Sandbox Code Playgroud) flake8安装后我无法运行pip install flake8.重新安装pep8没有帮助.我在Mac OS X 10.9.2上运行默认的python.我的pep8版本是1.5.6.
File "/usr/local/bin/flake8", line 5, in <module>
from pkg_resources import load_entry_point
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 2603, in <module>
working_set.require(__requires__)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 666, in require
needed = self.resolve(parse_requirements(requirements))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 565, in resolve
raise DistributionNotFound(req) # XXX put more info here
pkg_resources.DistributionNotFound: pep8>=1.4.6
Run Code Online (Sandbox Code Playgroud)