相关疑难解决方法(0)

我正在学习Python,并试图在下拉菜单中抓取此页面以获取特定值.之后,我需要单击结果表上的每个项目以检索特定信息.我能够选择项目并检索webdriver上的信息.但我不知道如何将响应URL传递给crawlspider.

driver = webdriver.Firefox()
driver.get('http://www.cppcc.gov.cn/CMS/icms/project1/cppcc/wylibary/wjWeiYuanList.jsp')
more_btn = WebDriverWait(driver, 20).until(
     EC.visibility_of_element_located((By.ID, '_button_select'))
            )  
more_btn.click()

## select specific value from the dropdown
driver.find_element_by_css_selector("select#tabJcwyxt_jiebie >     option[value='teyaoxgrs']").click()
driver.find_element_by_css_selector("select#tabJcwyxt_jieci > option[value='d11jie']").click()
search2 = driver.find_element_by_class_name('input_a2')
search2.click()
time.sleep(5)

## convert html to "nice format"
text_html=driver.page_source.encode('utf-8')
html_str=str(text_html)

## this is a hack that initiates a "TextResponse" object (taken from the Scrapy module)
resp_for_scrapy=TextResponse('none',200,{},html_str,[],None)

## convert html to "nice format"
text_html=driver.page_source.encode('utf-8')
html_str=str(text_html)

resp_for_scrapy=TextResponse('none',200,{},html_str,[],None)

所以这就是我被困住的地方.我能够使用上面的代码进行查询.但是如何将resp_for_scrapy传递给crawlspider？我把resp_for_scrapy替换为项目但是没有用.

## spider 
class ProfileSpider(CrawlSpider):
name …