小编use*_*803的帖子

在scrapy网络爬虫中获取错误

嗨我试着在我的代码中实现这个.但是我收到以下错误:exceptions.NameError: global name 'Request' is not defined.

from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector 
from bs4 import BeautifulSoup

class spider_aicte(BaseSpider):
    name = "Indian_Colleges"
    allowed_domains = ["http://www.domain.org"]
    start_urls = [
        "http://www.domain.org/appwebsite.html",
        ]

    def parse(self, response):
        filename = response.url.split("/")[-2]
        soup = BeautifulSoup(response.body)
        for link in soup.find_all('a'):
            download_link = link.get('href')
            if '.pdf' in download_link:
                pdf_link = "http://www.domain.org" + download_link
                print pdf_link
                class FileSpider(BaseSpider):
                    name = "fspider"
                    allowed_domains = ["www.domain.org"]
                    start_urls = [
                            pdf_link
                            ]
        for url in pdf_link:
            yield Request(url, …
Run Code Online (Sandbox Code Playgroud)

python web-crawler scrapy web-scraping scrapy-spider

4
推荐指数
1
解决办法
7389
查看次数