试图让Scrapy成为一个运行Crawl命令的项目

Question

我是Python和Scrapy的新手,我正在学习Scrapy教程.我已经能够通过使用DOS界面和键入来创建我的项目:

scrapy startproject dmoz

稍后教程将引用Crawl命令:

scrapy crawl dmoz.org

但每次我尝试运行时,我都会收到一条消息,告知这不是一个合法的命令.进一步展望它看起来我需要进入一个项目,这是我无法弄清楚的.我已经尝试将目录更改为我在startproject中创建的"dmoz"文件夹,但根本不识别Scrapy.

我确定我错过了一些明显的东西,我希望有人可以指出它.

Answer 1

你必须在'startproject'文件夹中执行它.如果找到scrapy.cfg文件,您将有另一个命令.你可以在这里看到差异:

$ scrapy startproject bar
$ cd bar/
$ ls
bar  scrapy.cfg
$ scrapy
Scrapy 0.12.0.2536 - project: bar

Usage:
  scrapy <command> [options] [args]

Available commands:
  crawl         Start crawling from a spider or URL
  deploy        Deploy project in Scrapyd target
  fetch         Fetch a URL using the Scrapy downloader
  genspider     Generate new spider using pre-defined templates
  list          List available spiders
  parse         Parse URL (using its spider) and print the results
  queue         Deprecated command. See Scrapyd documentation.
  runserver     Deprecated command. Use 'server' command instead
  runspider     Run a self-contained spider (without creating a project)
  server        Start Scrapyd server for this project
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

Use "scrapy <command> -h" to see more info about a command


$ cd ..
$ scrapy
Scrapy 0.12.0.2536 - no active project

Usage:
  scrapy <command> [options] [args]

Available commands:
  fetch         Fetch a URL using the Scrapy downloader
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

Use "scrapy <command> -h" to see more info about a command