tsc*_*wab 3 python pipeline file download scrapy
这是我的settings.py:
from scrapy.log import INFO
BOT_NAME = 'images'
SPIDER_MODULES = ['images.spiders']
NEWSPIDER_MODULE = 'images.spiders'
LOG_LEVEL = INFO
ITEM_PIPELINES = {
"images.pipelines.WritePipeline": 800
}
DOWNLOAD_DELAY = 0.5
Run Code Online (Sandbox Code Playgroud)
这是我的pipelines.py:
from scrapy import Request
from scrapy.pipelines.files import FilesPipeline
class WritePipeline(FilesPipeline):
def get_media_requests(self, item, info):
for url in item["file_urls"]:
yield Request(url)
def item_completed(self, results, item, info):
return item
Run Code Online (Sandbox Code Playgroud)
这是非常标准的,正常的东西。然而这是我的日志中的一行:
2015-06-25 18:16:41 [scrapy] INFO: Enabled item pipelines:
Run Code Online (Sandbox Code Playgroud)
So the pipeline is not enabled. What am I doing wrong here? I've used Scrapy a few times now, and I'm fairly positive the spider is fine. The item is just a normal item with file_urls and files.