Django+Vue前后端分离开发

未经许可请勿转载。
Please do not reprint this article without permission.

Project repository - NEURO-LEARN

Django+Vue+Celery

Environment

• For Django: Python, Django, MySQL, etc. Using pip to install modules including Django and MySQL is recommended;
• For Vue: Node.js. Using npm to install modules including Element-UI is recommended;
• For celery: rabbitmq, celery, django-celery. Using apt-get to install rabbitmq-server, and pip to install celery and django-celery;

Build Project

• Create project;

$ django-admin startproject neurolearn

• Create Django app as backend;

$ cd neurolearn
$ python manage.py startapp backend

• Install mysql if none installed, refering to installation of MySQL on Ubuntu18.04, solving access denied for user root@localhost, and creating a database;

'NAME': 'neurolearn'
'USER': 'root'
'PASSWORD': 'root'

• Change the default database to mysql and add backend to apps in settings.py;

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'neurolearn',
        'USER': 'root',
        'PASSWORD': 'root',
        'HOST': '127.0.0.1',
    }
}

• Initialize database and start server to test;

$ python manage.py makemigrations backend
$ python manage.py migrate
$ python manage.py runserver

• Install vue-cli to initialize a vue project;

$ npm install -g vue-cli
$ npm view vue-cli version // check the package version

• Create VueJS project as frontend;

$ vue-init webpack frontend

• Install vue dependencies and build the vue project;

$ cd frontend
$ npm install # install dependencies
$ npm run build # build project

• Add 'backend' to the INSTALLED_APPS in neurolearn/neurolearn/settings.py;

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'backend',
]

• Create superuser of django in order to use django admin.

$ python manage.py createsuperuser
# username: root
# email: leibingye@outlook.com
# password: root
# localhost:8000/admin/

References
整合Django+Vue.js框架快速搭建web项目
后端Django+前段Vue.js快速搭建项目
vue使用npm run build命令打包项目

Architecture Design

architecture_design_web.png

Integrate Vue into Django

• Configure url paths in neurolearn/neurolearn/urls.py;

from django.conf.urls import url
from django.contrib import admin
from django.views.generic import TemplateView

urlpatterns = [
    url(r'^admin/', admin.site.urls),
    url(r'^$', TemplateView.as_view(template_name="index.html")),
]

• Configure the 'DIRS' in neurolearn/neurolearn/settings.py;

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': ['frontend/dist'],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

• Change time zone and language code;

# LANGUAGE_CODE = 'en-us'
# TIME_ZONE = 'UTC'
LANGUAGE_CODE = 'zh-Hans'
TIME_ZONE = 'Asia/Shanghai'

• Add the path of static files rendered by django;

# add at the end of the file
STATICFILES_DIRS = [
    os.path.join(BASE_DIR, "frontend/dist/static")
]

• Run server to test frontend rendering by django;

$ python manage.py runserver

• Run the following command each time frontend is modified;

$ cd frontend
$ npm run build

• Using the following command allows debugging in Vue environment;

$ cd frontend
$ npm run dev

• Using Vue environment to visit Django API will result in cross-domain issues, one solution is using proxyTable in Vue, and the other is using django-cors-headers;

$ pip install django-cors-headers

• After installing django-cors-headers, we need to configure it in settings.py;

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'corsheaders.middleware.CorsMiddleware', # added
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

CORS_ORIGIN_ALLOW_ALL = True # added

References
Vue+Django+MySQL搭建指南（个人全栈快速开发）
我如何使用Django+Vue.js快速构建项目
Django与Vue之间的数据传递

Integrate Celery into Django

• Add 'djcelery' to the INSTALLED_APPS in neurolearn/neurolearn/settings.py;

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'backend',
    'djcelery',
]

• Configure RabbitMQ by add the following codes at the end of neurolearn/neurolearn/settings.py;

import djcelery
djcelery.setup_loader()

BROKER_HOST = "localhost"
BROKER_PORT = 5672
BROKER_USER = "guest"
BROKER_PASSWORD = "guest"
BROKER_VHOST = "/"

• Running $ python manage.py will result in several new commands to control worker in celery;

[djcelery]
    celery
    celerybeat
    celerycam
    celeryd
    celeryd_detach
    celeryd_multi
    celerymon
    djcelerymon

• Add a file backend/tasks.py;

from celery.decorators import task

@task
def add(x, y):
  return x + y
# @ is the decorator, making the add function a callback function
# when calling add in a webapp, the add function doesn't execute immediately
# instead the function name and parameters will be packed as a message and
# sent to the queue, then the worker will process these messages one by one

• Create a worker awaiting task messages;

$ python manage.py migrate
$ python manage.py celeryd -l info

• Open another console and use the following command to open interactive console;

$ python manage.py shell
>>> from backend.tasks import add
>>> r = add.delay(3, 5)
>>> r.wait()
8

References
使用django+celery+RabbitMQ实现异步执行

---

NEURO-LEARN-WEB

UI Design

User Interface

Vue and Element-UI

• Refer to official site for installation and usage guide;
• Use template from this repository, which looks like this;
• Replace frontend in the project with template, type the following commands to build the vue project, integrating it into django framework;

$ npm install
$ npm install node-sass // if needed
$ npm run dev // serve with hot reload at localhost:8080
$ npm run build // build for Django to serve at localhost:8000

• To customize navigation menu and router, change the code in NavMenu.vue to configure the navigation, routes.js to configure the router, and index.vue in each page in pages to configure the template;
• The pages folder consists of home (routed by NEURO-LEARN title), overview, newtask, submissions, viewer, and help, which are routed by items in NavMenu except for home;

Note: 
The template is develped using element-ui 1.4, which is out of date. Use element-ui 2.7 when develop frontend.

• The way to pass eslint check is to add a comment like below at the end of the code;

//eslint-disable-line

• To use scss, install node-sass and sass-loader;

$ npm install --save-dev sass-loader style-loader css-loader

• To change the theme colors in element-ui, refer to this site for help;
• When defining the style of a page by css, name the class carefully or use a nested css since it is effective across files;
• Use this.$router.replace to realize in-page redireting;

this.$router.replace({
  path: '/submissions',
  component: resolve => require(['@/pages/analysis/submissions'], resolve)
})

• Use this.$router.go(0) to realize in-page refreshing;

References
Element-UI Documentation

Data Transaction

• Refer to django_with_vue and django-vue for examples of using axios and database for data transaction between Vue and Django;
• As mentioned above, the configuration of databases is in the settings.py, and by default the name of table created by models.py is 'appname_modelclassname', in which the modelclassname refer to the class defined in models.py;
• Models are called and instantiated by views.py, which received the http request from frontend and return a response;
• The urls of functions in views are defined in urls.py， which is included in the urls.py in project folder;
• The urls.py in project folder contains the urls when frontend sends request to http://127.0.0.1:8000/;
• As $http in vue requires an out-of-date module named vue-resource, it is recommended to use axios instead;
• Using axios and django JsonResponse:
  • Excecute POST request with axios to submit a task:

  // import axios from 'axios'
  axios.post('http://127.0.0.1:8000/api/new_task', JSON.stringify(this.newform))
        .then(response => {
          var res = response.data
          if (res.error_num === 0) {
            this.$router.replace({
              path: '/submissions',
              component: resolve => require(['@/pages/analysis/submissions'], resolve)
            })
            console.log(res)
          } else {
            this.$message.error('Failed submission!')
            console.log(res['msg'])
          }
        })
  

  Note: the 'newform' object defined in data object in vue can't be within any other object, as it may not satisfy two-way data-binding with v-model.
  

  • Excecute GET request with axios to get all submissions list:

  // import axios from 'axios'
  axios.get('http://127.0.0.1:8000/api/show_submissions')
        .then(response => {
          var res = response.data
          if (res.error_num === 0) {
            console.log(res)
            this.submissions_table = res['list']
            console.log(res['list'])
          } else {
            this.$message.error('Failed!')
            console.log(res['msg'])
          }
        })
  

  Note: axios support promise, which is the function().then.then ... structure.
  

  • Uniform Resource Locator is defined in urls in backend app, which is included in urls in neurolearn_dev project:

  # neurolearn_dev/urls.py
  from django.conf.urls import url, include
  from django.contrib import admin
  from django.views.generic import TemplateView
  import backend.urls
  
  urlpatterns = [
      url(r'^admin/', admin.site.urls),
      url(r'^api/', include(backend.urls)), # added
      url(r'^$', TemplateView.as_view(template_name="index.html")),
  ]
  
  # backend/urls.py
  from django.conf.urls import url, include
  from . import views
  
  urlpatterns = [
      url(r'add_book$', views.add_book, ),
      url(r'show_books$', views.show_books, ),
      url(r'new_task$', views.new_task, ),
      url(r'show_submissions$', views.show_submissions, ),
      ]
  

  • Receive request and generate response in views inside django:

  @require_http_methods(["POST"])
  def new_task(request):
      response = {}
      postBody = json.loads(request.body)
      try:
          task = Submissions_Demo(
              task_name=postBody.get('task_name'),
              task_type=postBody.get('task_type'),
              train_data=postBody.get('train_data'),
              test_data=postBody.get('test_data'),
              label=postBody.get('label'),
              feat_sel=postBody.get('feat_sel'),
              estimator=postBody.get('estimator'),
              cv_type=postBody.get('cv_type'),
              note=postBody.get('note'),
              verbose=postBody.get('verbose'),
              task_status='Submitted',
              task_result=''
          )
          task.save()
          response['post_body'] = postBody
          response['msg'] = 'success'
          response['error_num'] = 0
      except Exception as e:
          response['post_body'] = postBody
          response['msg'] = str(e)
          response['error_num'] = 1
  
      return JsonResponse(response)
  
  @require_http_methods(["GET"])
  def show_submissions(request):
      response = {}
      try:
          submissions = Submissions_Demo.objects.filter()
          response['list']  = json.loads(serializers.serialize("json", submissions))
          response['msg'] = 'success'
          response['error_num'] = 0
      except  Exception as e:
          response['msg'] = str(e)
          response['error_num'] = 1
  
      return JsonResponse(response)
  

  • To initialize and manipulate databases, use models in django:

  class Submissions_Demo(models.Model):
      task_id = models.DateTimeField('Edit the date', auto_now=True)
      task_name = models.CharField(max_length=64)
      task_type = models.CharField(max_length=64)
      train_data = models.CharField(max_length=64)
      test_data = models.CharField(max_length=64)
      label = models.CharField(max_length=64)
      feat_sel = models.CharField(max_length=64)
      estimator = models.CharField(max_length=64)
      cv_type = models.CharField(max_length=64)
      note = models.CharField(max_length=64)
      verbose = models.BooleanField()
      task_status = models.CharField(max_length=64)
      task_result = models.CharField(max_length=1024)
  
      def __unicode__(self):
          return self.task_id
  

  Note: after each altering of models, type 'python manage.py makemigrations' and 'python manage.py migrate' in commandline to alter databases.
  

• View databases and tables inside them:

# mysql -u root -p
# password: root
mysql> use neurolearn_dev;
mysql> select * from backend_submissions_demo

references
Django模型Model自定义表名和字段列名
axios全攻略
vue $http请求服务
Vue:axios中的POST请求传参问题
Vue + Django

File Uploading

• Refer to this blog for the tutorial of using Django forms to realize file uploading;
• First add an api redirecting to the view funciton upload_data;

url(r'upload_data$', views.upload_data, ),

• Add a table in the database containing data information;

class Data_Demo(models.Model):
    data_id = models.DateTimeField('Edit the date', auto_now=True)
    data_name = models.CharField(max_length=64)
    data_path = models.CharField(max_length=128)

    def __unicode__(self):
        return self.task_id

• Add view funciton to process request and save uploaded file;

@require_http_methods(['POST'])
def upload_data(request):
    response = {}
    try:
        obj = request.FILES.get('test')
        data = Data_Demo()
        data.data_name = 'test'
        data.data_path = obj.name
        data.save()
        handle_uploaded_file(obj)
        
        response['msg'] = 'success'
        response['error_num'] = 0
    
    except Exception as e:
        response['msg'] = str(e)
        response['error_num'] = 1

    return JsonResponse(response)

def handle_uploaded_file(f):
    try:
        path = 'data/'
        if not os.path.exists(path):
            os.makedirs(path)
        else:
            file_name = str(path + f.name)
            destination = open(file_name, 'wb+')
            for chunk in f.chunks():
                destination.write(chunk)
            destination.close()
    except Exception as e:
        print(e)
    return f.name, path

• Create a front-end template to upload file;

<template>
  <div style="background-color: #FFFFFF; margin: 14px; padding: 14px">
    <el-upload
      class="upload-demo"
      action="http://127.0.0.1:8000/api/upload_data"
      name="test"
      :on-change="handleChange"
      :on-success="uploadSuccess"
      :file-list="fileList">
      <el-button size="small" type="primary">点击上传</el-button>
      <div slot="tip" class="el-upload__tip">只能上传jpg/png文件，且不超过500kb</div>
    </el-upload>
  </div>
</template>

<script>
export default {
  data () {
    return {
      fileList: []
    }
  },
  methods: {
    handleChange (file, fileList) {
      this.fileList = fileList.slice(-4)
    },
    uploadSuccess (response) {
      console.log(response.error_num, response.msg)
    }
  }
}
</script>

references
Django快速实现文件上传
Element-UI Upload上传
Django多文件上传，只能保存最后一个文件
Django文件上传到后台的三种方式

File Downloading

• Refer to this site for three ways of realizing file downloading in Django;

@require_http_methods(["GET"])
def download_templates(request):
    response = {}
    template_type = request.GET.get('template_type')

    template_file=open('templates/' + template_type + '.zip', 'rb')
    response =FileResponse(template_file)
    response['Content-Type']='application/octet-stream'
    response['Content-Disposition']='attachment;filename=\"' + template_type + '.zip\"'
    return response

<a href="http://127.0.0.1:8000/api/download_templates?template_type=dataset_templates"></a>

Visualization

• Refer to this site for the implementation of image transaction, to be specific,

import io
from PIL import Image

@require_http_methods(["GET"])
def show_roc(request):
    response = {}
    task_id = request.GET.get('task_id')

    buf = io.BytesIO()
    img = Image.open('results/' + task_id + '/190514_ROC_curve_rfe_svm_test_data.png')
    img.save(buf, 'png')

    return HttpResponse(buf.getvalue(), 'image/png')

<img :src="'http://127.0.0.1:8000/api/show_roc?task_id=' + this.taskid" style="width: 700px">

• To use this taskid, we have to realize parameter passing between modules (in this case, submissions and viewer) using v-router;

// Source module
onRowClick (row) {
      this.$router.push({
        path: '/analysis/viewer',
        query: {taskid: row.fields.task_id}
      })
    }

// Destination module
taskid: this.$route.query.taskid,

Service

Celery and RabbitMQ

• Refer to this site as before to get the basic idea of celery workers;
• Define own tasks;

@task
def new_ml_task():
    test_task()
    return

• Call the task in views.py;

@require_http_methods(["POST"])
def new_task(request):
    response = {}
    postBody = json.loads(request.body)
    try:
        task = Submissions_Demo(
            task_name=postBody.get('task_name'),
            task_type=postBody.get('task_type'),
            train_data=postBody.get('train_data'),
            test_data=postBody.get('test_data'),
            label=postBody.get('label'),
            feat_sel=postBody.get('feat_sel'),
            estimator=postBody.get('estimator'),
            cv_type=postBody.get('cv_type'),
            note=postBody.get('note'),
            verbose=postBody.get('verbose'),
            task_status='Submitted',
            task_result=''
        )
        task.save()

        # create new celery task
        new_ml_task.delay()
        ...

• Initiate workers using the following command;

$ python manage.py celeryd -l info

Databases

• To use Django-Model to manipulate databases, refer to this site;
• To visualize the database in a descending order, which means the frontmost item being the most recent one, refer to this site

Deployment

Local Server

• Refer to this site and this site to deploy a Django+Vue project on production environment;
• If the console of the browser shows something like 'Resource interpreted as Stylesheet but transferred with MIME type text/plain', refer to this site to solve it.

GitHub

• To deploy a repository on GitHub, refer to this site, basically after creating a project, pull a branch from it and replace all files with your own, then just commit;