Scrapy

scrapy -h

查看常见的命令

Available commands:
  bench         Run quick benchmark test
  fetch         Fetch a URL using the Scrapy downloader
  genspider     Generate new spider using pre-defined templates
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

fetch 命令：不依赖项目直接爬取网页命令，显示爬虫爬取的过程

image.png

不显示日志：

fetch -- none log

#用scrapy view命令下载百度网页，并用浏览器打开这个问题
scrapy  view  http://www.baidu.com