docker搭建数据查询及可视化工具redash

作者: Hero_seven | 来源:发表于2020-03-14 23:09 被阅读0次

docker搭建数据查询及可视化工具redash
Team 报表系统定制——基于 Redash 的二次开发
Redash详细部署教程
MySQL基础语法操作
使用 Power BI Desktop 模型作为 SSAS 服务
docker container monitor
数据分析师之路(一)
使用Docker搭建Postgresql数据库
Elasticsearch-head插件安装
docker上搭建可视化实时监测

1. redash简介

redash是一款开源的数据查询及可视化的工具，可连接各类数据库，支持丰富的数据库类型；可通过编写SQL查询需求数据，并对数据进行可视化展示，redash支持绘制各种图表，如柱状图、曲线图、饼图、数据透视表等等。
redash的官方使用指南

2. docker搭建redash(ubuntu18.04)

官方文档

(1) docker-compose.yml

/opt/redash 为redash的工作目录，可自定义进行更改。

version: "3.4"
x-redash-service: &redash-service
  image: redash/redash:8.0.0.b32245
  depends_on:
    - postgres
    - redis
  env_file: /opt/redash/env
  restart: always
services:
  server:
    <<: *redash-service
    command: server
    ports:
      - "5000:5000"
    environment:
      REDASH_WEB_WORKERS: 4
  scheduler:
    <<: *redash-service
    command: scheduler
    environment:
      QUEUES: "celery"
      WORKERS_COUNT: 1
  scheduled_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "scheduled_queries,schemas"
      WORKERS_COUNT: 1
  adhoc_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "queries"
      WORKERS_COUNT: 2
  redis:
    image: redis:5.0-alpine
    restart: always
  postgres:
    image: postgres:9.6-alpine
    env_file: /opt/redash/env
    volumes:
      - /opt/redash/postgres-data:/var/lib/postgresql/data
    restart: always
  nginx:
    image: redash/nginx:latest
    ports:
      - "80:80"
    depends_on:
      - server
    links:
      - server:redash
    restart: always

(2) 设置环境变量env文件

通过环境变量的方式，定义系统运行所需要的一些配置。
注意：更新配置后请重新启动服务进行生效。

PYTHONUNBUFFERED=0
REDASH_LOG_LEVEL=INFO
REDASH_REDIS_URL=redis://redis:6379/0
POSTGRES_PASSWORD=wxK3u5spuoqI5gaogYdz0yMTkKtZilYy
REDASH_COOKIE_SECRET=xvyXYDAYgMNB1hbjJPrha95LkeXLArFP
REDASH_SECRET_KEY=lfLqjWNNetKNU4gb1uTwPxukNPrunuey
REDASH_DATABASE_URL=postgresql://postgres:wxK3u5spuoqI5gaogYdz0yMTkKtZilYy@postgres/postgres

# 邮件相关配置
REDASH_MAIL_SERVER=smtp.exmail.qq.com
REDASH_MAIL_PORT=587
REDASH_MAIL_USERNAME=example@example.com
REDASH_MAIL_PASSWORD=password
REDASH_MAIL_DEFAULT_SENDER=发送者名称<example@example.com>
REDASH_MAIL_USE_TLS=true

(3) 创建数据库

docker-compose -f docker-compose.yml run --rm server create_db

(4) 启动服务

docker-compose up -d

(5) 测试邮件功能

通过以下命令，可测试是否成功配置邮件，若配置成功，则设置的邮箱地址可接收到自己发送的一封邮件。

docker-compose run --rm server manage send_test_mail

(6) 使用脚本进行自动搭建部署redash

通过编写脚本setup.sh，来完成以上1-5的步骤，自动进行搭建部署redash。
运行setup.sh即可完成相关软件的安装、环境变量的生成、以及服务的启动等操作。

#!/usr/bin/env bash
# This script setups dockerized Redash on Ubuntu 18.04.
set -eu

REDASH_BASE_PATH=/opt/redash

install_docker(){
    # Install Docker
    export DEBIAN_FRONTEND=noninteractive
    sudo apt-get -qqy update
    DEBIAN_FRONTEND=noninteractive sudo -E apt-get -qqy -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade 
    sudo apt-get -yy install apt-transport-https ca-certificates curl software-properties-common wget pwgen
    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
    sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
    sudo apt-get update && sudo apt-get -y install docker-ce

    # Install Docker Compose 这里使用国内镜像源进行安装，速度更快
    # sudo curl -L https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
    sudo curl -L https://get.daocloud.io/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
    sudo chmod +x /usr/local/bin/docker-compose

    # Allow current user to run Docker commands
    sudo usermod -aG docker $USER
}

create_directories() {
    if [[ ! -e $REDASH_BASE_PATH ]]; then
        sudo mkdir -p $REDASH_BASE_PATH
        sudo chown $USER:$USER $REDASH_BASE_PATH
    fi

    if [[ ! -e $REDASH_BASE_PATH/postgres-data ]]; then
        mkdir $REDASH_BASE_PATH/postgres-data
    fi
}

create_config() {
    if [[ -e $REDASH_BASE_PATH/env ]]; then
        rm $REDASH_BASE_PATH/env
        touch $REDASH_BASE_PATH/env
    fi

    COOKIE_SECRET=$(pwgen -1s 32)
    SECRET_KEY=$(pwgen -1s 32)
    POSTGRES_PASSWORD=$(pwgen -1s 32)
    REDASH_DATABASE_URL="postgresql://postgres:${POSTGRES_PASSWORD}@postgres/postgres"

    echo "PYTHONUNBUFFERED=0" >> $REDASH_BASE_PATH/env
    echo "REDASH_LOG_LEVEL=INFO" >> $REDASH_BASE_PATH/env
    echo "REDASH_REDIS_URL=redis://redis:6379/0" >> $REDASH_BASE_PATH/env
    echo "POSTGRES_PASSWORD=$POSTGRES_PASSWORD" >> $REDASH_BASE_PATH/env
    echo "REDASH_COOKIE_SECRET=$COOKIE_SECRET" >> $REDASH_BASE_PATH/env
    echo "REDASH_SECRET_KEY=$SECRET_KEY" >> $REDASH_BASE_PATH/env
    echo "REDASH_DATABASE_URL=$REDASH_DATABASE_URL" >> $REDASH_BASE_PATH/env
}

setup_compose() {
    REQUESTED_CHANNEL=stable
    LATEST_VERSION=`curl -s "https://version.redash.io/api/releases?channel=$REQUESTED_CHANNEL"  | json_pp  | grep "docker_image" | head -n 1 | awk 'BEGIN{FS=":"}{print $3}' | awk 'BEGIN{FS="\""}{print $1}'`

    cd $REDASH_BASE_PATH
    GIT_BRANCH="${REDASH_BRANCH:-master}" # Default branch/version to master if not specified in REDASH_BRANCH env var
    wget https://raw.githubusercontent.com/getredash/setup/${GIT_BRANCH}/data/docker-compose.yml
    sed -ri "s/image: redash\/redash:([A-Za-z0-9.-]*)/image: redash\/redash:$LATEST_VERSION/" docker-compose.yml
    echo "export COMPOSE_PROJECT_NAME=redash" >> ~/.profile
    echo "export COMPOSE_FILE=/opt/redash/docker-compose.yml" >> ~/.profile
    export COMPOSE_PROJECT_NAME=redash
    export COMPOSE_FILE=/opt/redash/docker-compose.yml
    sudo docker-compose run --rm server create_db
    sudo docker-compose up -d
}

install_docker
create_directories
create_config
setup_compose