小编Pat*_*ein的帖子

我的扩展中未收到 Scrapy spider_idle 信号

spider_idle在接收信号时的几个蜘蛛之间有共同的行为,我想将此行为移动到扩展中。

我的分机已经成功侦听spider_openedspider_closed发出信号。但是,spider_idle没有接收到信号。

这是我的扩展程序(为简洁起见进行了编辑):

import logging
import MySQLdb
import MySQLdb.cursors
from scrapy import signals

logger = logging.getLogger(__name__)

class MyExtension(object):

    def __init__(self, settings, stats):
        self.settings = settings
        self.stats = stats

    @classmethod
    def from_crawler(cls, crawler):

        # instantiate the extension object
        ext = cls(crawler.settings, crawler.stats)

        # connect the extension object to signals
        crawler.signals.connect(ext.spider_opened, signal=signals.spider_opened)
        crawler.signals.connect(ext.spider_closed, signal=signals.spider_closed)
        crawler.signals.connect(ext.spider_idle, signal=signals.spider_idle)

        # return the extension object
        return ext

    def spider_opened(self, spider):
        logger.info("start logging spider %s", spider.name)

    def spider_closed(self, spider): …
Run Code Online (Sandbox Code Playgroud)

scrapy scrapy-signal

5
推荐指数
0
解决办法
576
查看次数

标签 统计

scrapy ×1

scrapy-signal ×1