我正在使用以下代码来抓取该网站(http://profiles.ehs.state.ma.us/Profiles/Pages/ChooseAPhysician.aspx?Page=1);但是,获取以下TypeError:
“文件“ C:\ Users \ Anaconda2 \ lib \ site-packages \ scrapy \ contrib \ spiders \ crawl.py”,第83行,位于_compile_rules self._rules = [self.rules中的r的copy.copy(r)] TypeError:“ Rule”对象不可迭代”
我在第83行上没有编写任何代码,因此想知道是否有人对如何解决此问题有想法?我在Windows中使用Python 2.7。
谢谢!
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import scrapy
from scrapy.contrib.spiders import CrawlSpider, Rule
from scrapy.contrib.linkextractors import LinkExtractor
from scrapy.selector import HtmlXPathSelector
class MdiMassSpider(CrawlSpider):
name = "MdiMass"
allowed_domains = ["http://profiles.ehs.state.ma.us/Profiles/Pages/FindAPhysician.aspx"]
start_urls = ["http://profiles.ehs.state.ma.us/Profiles/Pages/ChooseAPhysician.aspx?Page=1"]
driver = webdriver.Chrome()
rules …Run Code Online (Sandbox Code Playgroud)