美文网首页
crawlab latest 部署

crawlab latest 部署

作者: SodaCrush | 来源:发表于2024-08-29 17:36 被阅读0次

docker-compose up -d

docker-compose.yml 如下:

version: '3.3'
services:
  master:
    image: crawlabteam/crawlab
    container_name: crawlab_master
    restart: always
    environment:
      CRAWLAB_NODE_MASTER: Y  # Y: master node
      CRAWLAB_MONGO_HOST: mongo  # mongo host address. In the docker compose network, directly refer to the service name
      CRAWLAB_MONGO_PORT: 27017  # mongo port 
      CRAWLAB_MONGO_DB: crawlab  # mongo database 
      CRAWLAB_MONGO_USERNAME: username  # mongo username
      CRAWLAB_MONGO_PASSWORD: password  # mongo password 
      CRAWLAB_MONGO_AUTHSOURCE: admin  # mongo auth source 
      CRAWLAB_TASK_WORKERS: 800  # number of task executors 任务执行器个数(并行执行任务数)
      CRAWLAB_RPC_WORKERS: 1600  # number of RPC workers RPC 工作协程个数
    volumes:
      - "/opt/.crawlab/master:/root/.crawlab"  # 持久化 crawlab 元数据
      - "/opt/crawlab/master:/data"  # 持久化 crawlab 数据
      - "/var/crawlab/log:/var/log/crawlab" # 持久化 crawlab 任务日志
      - "/data/crawlab_workspace:/root/crawlab_workspace/"  # 持久化爬虫项目文件到宿主机本地
    ports:
      - "6080:8080"  # exposed api port
    depends_on:
      - mongo

  mongo:
    image: mongo:4.2
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: username  # mongo username
      MONGO_INITDB_ROOT_PASSWORD: password  # mongo password
    volumes:
      - "/opt/crawlab/mongo/data/db:/data/db"  # persistent mongo data
    ports:
      - "27017:27017"  # expose mongo port to host machine

相关文章

网友评论

      本文标题:crawlab latest 部署

      本文链接:https://www.haomeiwen.com/subject/civxljtx.html