Ubuntu16.04中Scrapyd 安装与配置

• pip安装scrapyd：

pip install scrapyd

安装完毕之后，Scrapyd在运行时需要读取/etc/scrapyd/scrapyd.conf文件，由于在Scrapyd 1.2 版本之后不会自动创建/etc/scrapyd/scrapyd.conf文件，需要我们自行添加。

• 配置scrapyd.conf

sudo mkdir /etc/scrapyd
sudo vim /etc/scrapyd/scrapyd.conf

• 接着写入如下内容

[scrapyd]
eggs_dir    = eggs
logs_dir    = logs
items_dir   =
jobs_to_keep = 5
dbs_dir     = dbs
max_proc    = 0
max_proc_per_cpu = 10 #默认为4
finished_to_keep = 100
poll_interval = 5.0
bind_address = 0.0.0.0 #默认为127.0.0.1，修改后使外网可以访问
http_port   = 6800
debug       = off
runner      = scrapyd.runner
application = scrapyd.app.application
launcher    = scrapyd.launcher.Launcher
webroot     = scrapyd.website.Root

[services]
schedule.json     = scrapyd.webservice.Schedule
cancel.json       = scrapyd.webservice.Cancel
addversion.json   = scrapyd.webservice.AddVersion
listprojects.json = scrapyd.webservice.ListProjects
listversions.json = scrapyd.webservice.ListVersions
listspiders.json  = scrapyd.webservice.ListSpiders
delproject.json   = scrapyd.webservice.DeleteProject
delversion.json   = scrapyd.webservice.DeleteVersion
listjobs.json     = scrapyd.webservice.ListJobs
daemonstatus.json = scrapyd.webservice.DaemonStatus

• 配置内容参考网址

https://scrapyd.readthedocs.io/en/stable/config.html#example-configuration-file