小编The*_*ord的帖子

我正在从脚本运行scrapy,但它所做的就是激活蜘蛛.它不会通过我的项目管道.我读过http://scrapy.readthedocs.org/en/latest/topics/practices.html,但它没有说出包含管道的任何内容.

我的设置:

Scraper/
    scrapy.cfg
    ScrapyScript.py
    Scraper/
        __init__.py
        items.py
        pipelines.py
        settings.py
        spiders/
            __init__.py
            my_spider.py

我的剧本:

from twisted.internet import reactor
from scrapy.crawler import Crawler
from scrapy.settings import Settings
from scrapy import log, signals
from Scraper.spiders.my_spider import MySpiderSpider

spider = MySpiderSpider(domain='myDomain.com')
settings = get_project_settings
crawler = Crawler(Settings())
crawler.signals.connect(reactor.stop, signal=signals.spider_closed)
crawler.configure()
crawler.crawl(spider)
crawler.start()
log.start()
log.msg('Reactor activated...')
reactor.run()
log.msg('Reactor stopped.')

我的管道:

from scrapy.exceptions import DropItem
from scrapy import log
import sqlite3


class ImageCheckPipeline(object):

    def process_item(self, item, spider):
        if item['image']:
            log.msg("Item added successfully.")
            return …