scrapy-splash框架做爬虫（1）

环境 centos-8 , python3.7.7

安装docker

sudo yum remove docker \

                  docker-client \

                  docker-client-latest \

                  docker-common \

                  docker-latest \

                  docker-latest-logrotate \

                  docker-logrotate \

                  docker-engine

sudo yum install -y yum-utils \

  device-mapper-persistent-data \

  lvm2

sudo yum-config-manager \

    --add-repo \

    http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

sudo yum install docker-ce docker-ce-cli containerd.io

docker 换源

sudo vi /etc/docker/daemon.json

{

   "registry-mirrors":[

        "http://docker.mirrors.ustc.edu.cn",

        "http://hub-mirror.c.163.com",

        "http://registry.docker-cn.com"

    ] ,

   "insecure-registries":[

      "docker.mirrors.ustc.edu.cn",

        "registry.docker-cn.com"

    ]

}     

国内镜像有

docker中国官方

https://registry.docker-cn.com

网易

http://hub-mirror.c.163.com

USTC

http://docker.mirrors.ustc.edu.cn

阿里云

http://<你的ID>.mirror.aliyuncs.com

docker启动

service docker restart

sudodocker pull scrapinghub/splash

sudo docker run -it -p 8050:8050 --rm scrapinghub/splash

安装 scrapy-splash 

pip3 install scrapy

pip3 install scrapy-splash

到此前置基本齐全了