Django djcelery gevent uwsgi版本适应

Linux环境

介绍

异步/定时任务在django中必不可少，如果此时又集成了gevent相关的第三方包，可能会出现各种问题。
众所周知gevent包比较霸道，它是直接启动协程，并且替换了很多基本包的逻辑。
因此在这里提供了一组可以正常运行的包的版本依赖，这是编者google了好几天的成果。。

requirements.txt

Django == 2.2.11
djangorestframework == 3.10.3
django-celery == 3.2.2
uwsgi == 2.0.18
gevent==1.2.2
idna

manage.py，增加代码

from gevent import monkey
monkey.patch_all()

uwsgi.ini，增加配置

gevent=100                                   ;
gevent-early-monkey-patch=true  ;

有@shared_task装饰器的文件中增加

import django
django.setup()

settings.py文件djcelery配置

# Celery
# ------------------------------------------------------------------------------
import djcelery
djcelery.setup_loader()
if USE_TZ:
    # http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-timezone
    CELERY_TIMEZONE = TIME_ZONE
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-broker_url
BROKER_URL = env.str("CELERY_BROKER_URL")
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-result_backend
CELERY_RESULT_BACKEND = BROKER_URL
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-accept_content
CELERY_ACCEPT_CONTENT = ["json"]
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-task_serializer
CELERY_TASK_SERIALIZER = "json"
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-result_serializer
CELERY_RESULT_SERIALIZER = "json"
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#task-time-limit
CELERY_TASK_TIME_LIMIT = 4*60*60 # 单个任务最大运行时间
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#task-soft-time-limit
# CELERY_TASK_SOFT_TIME_LIMIT = 60
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#beat-scheduler
CELERY_ENABLE_UTC = True
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
CELERY_RESULT_BACKEND = 'djcelery.backends.database:DatabaseBackend'

CELERY_TASK_RESULT_EXPIRES = 1800
CELERYD_CONCURRENCY = 1 if DEBUG else 4  # 并发的worker数量
CELERYD_MAX_TASKS_PER_CHILD = 20  # 每个worker最多执行100次任务被销毁，防止内存泄漏
CELERY_FORCE_EXECV = True  # 有些情况可以防止死锁

启动文件start.sh

#!/usr/bin/env bash
# delete celery pid file
rm celerybeat.pid w1.pid
# start celery worker
celery multi start w1 -A config.celery_app -l info -P gevent --logfile=./logs/worker.log
# start celery beat
nohup python3 manage.py celery beat -l info > ./logs/beat.log 2>&1 &
# makemigrations
python3 manage.py makemigrations
# migrate
python3 manage.py migrate
# start Unchained
uwsgi --ini ./config/uwsgi.ini