pip3 install scrapyd(服务)
pip3 install scrapyd-client(打包)
scrapyd-deploy -p xiachufang --version 1.0
需要安装curl
sudo apt-get install curl
调度爬虫开始运行
curl http://localhost:6800/schedule.json -d project=xiachufang -d spider=xcfCrawlSpider
关闭爬虫
curl http://localhost:6800/cancel.json -d project=xiachufang -d job=f9b3db46130a11e98490bca8a6e07fd6 (jobid)
获取部署的项目列表
curl http://localhost:6800/listprojects.jsoncurl http://localhost:6800/listversions.json?project=xiachufang
获取项目的版本号
curl http://localhost:6800/listversions.json?project=xiachufang
获取项目下的爬虫文件
curl http://localhost:6800/listspiders.json?project=myproject
获取爬虫的运行状态
curl http://localhost:6800/listjobs.json?project=myproject
删除对应版本的项目
curl http://localhost:6800/delversion.json -d project=myproject -d version=r99
直接删除项目
curl http://localhost:6800/delproject.json -d project=myproject
网友评论