不确定我是否有Scrapy或Twisted或??? 通过easy_install运行Python 2.7 32位和最新的scrapy/dependencies
尝试模拟POST请求登录并使其适合工作.回到Scrapy的教程来做这件事并且它不起作用.
这是蜘蛛代码:
from scrapy.spider import BaseSpider
class LoginSpider(BaseSpider):
name = 'example.com'
start_urls = ['http://www.example.com/users/login.php']
def parse(self, response):
return [FormRequest.from_response(response,
formdata={'username': 'john', 'password': 'secret'},
callback=self.after_login)]
def after_login(self, response):
# check login succeed before going on
if "authentication failed" in response.body:
self.log("Login failed", level=log.ERROR)
return
# continue scraping with authenticated session...
_________________________________________
结果如下:
C:\Users\jhoiland\scrapy\tutorial>scrapy crawl example.com
2012-01-23 18:59:39-0800 [scrapy] INFO: Scrapy 0.14.1 started (bot: tutorial)
2012-01-23 18:59:39-0800 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetCon
sole, …