小编fcm*_*max的帖子

Framework Scrapy - Scrapyd服务器.

我在蜘蛛中获得jobid值有一些问题.

将数据发布到http:// localhost:6800/schedule.json后,响应为

status = ok
jobid = bc2096406b3011e1a2d0005056c00008

但是我需要在进程中使用当前蜘蛛内部的这个jobid.它可用于open {jobid} .log文件或其他动态原因.

class SomeSpider(BaseSpider):
    name = "some"
    start_urls = ["http://www.example.com/"]
    def parse(self, response):
        items = []
        for val in values:
            item = SomeItem()
            item['jobid'] = self.jobid # ???!
            items.append(item)
        return items

但是只有在完成任务后我才会看到这个工作:(谢谢!

我无法在parse方法中更改spider设置.但绝对必须是一种方式.

例如:

class SomeSpider(BaseSpider):
    name = 'mySpider'
    allowed_domains = ['example.com']
    start_urls = ['http://example.com']
    settings.overrides['ITEM_PIPELINES'] = ['myproject.pipelines.FirstPipeline']
    print settings['ITEM_PIPELINES'][0]
    #printed 'myproject.pipelines.FirstPipeline'
    def parse(self, response):
        #...some code
        settings.overrides['ITEM_PIPELINES'] = ['myproject.pipelines.SecondPipeline']
        print settings['ITEM_PIPELINES'][0]
        # printed 'myproject.pipelines.SecondPipeline'
        item = Myitem()
        item['mame'] = 'Name for SecondPipeline'  

但!项目将由FirstPipeline处理.新的ITEM_PIPELINES参数不起作用.如何在开始抓取后更改设置？提前致谢!