docker系统架构|系统架构与docker的完美整合|可升级系

需要对gitlab的系统架构有一定的了解，本文不讲解如何搭建gitlab，默认都会,本文所用例子配置都是示例，仅供参考学习。

1.首先很重要的一步，也是必须花多一点的时间思考：梳理清楚你整个项目中涉及的配置文件、数据存储所用的server、项目中的各项vendor、系统层面的service、log管理

2.接下来就是考虑版本迭代时如何数据同步的问题，做到每次版本更新只需要换一个docker镜像就行，搭配上系统架构的.sh，就实现了一键化部署

。。。此处思考过后，根据不同的方式进行划分，而我采用的是联系性关系进行划分，大体上划分为3个层面。

• 容器内配置
• 容器外映射配置
• 映射内外交互配置(初始化项目配置)

之后选择你实现这个docker容器的方式，Dockerfile|手动docker pull/push

手动docker pull/push 这里就不手动了，咱们直接来讲DockerFile

• Dockerfile，目录结构就不列出来了

记得同步docker容器时间与宿主机的时间

#ubuntu
RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
#centOS

FROM ubuntu:16.04
MAINTAINER leeboot  <leeboot@163.com>

SHELL ["/bin/bash", "-c"],
ENV LANG C.UTF-8
COPY assets /root/assets/
#update and install java8 and required tools
RUN cd /etc/apt/ && /bin/cp -rf /root/assets/source.list /etc/apt/ && \
    apt update -q \
        && DEBIAN_FRONTEND=noninteractive && apt install -yq --no-install-recommends \
        vim unzip libbz2-dev bzip2 gcc g++ make openjdk-8-jre-headless libmozjs-24-0v5 libmozjs-24-dev \
        memcached nginx postgresql \
        php php-dev php-mbstring php-mcrypt php-mysql php-mysqli php-pgsql php-zip php-curl php-ctype \
        php-bz2 php-dom php-gd php-gmp php-xml php-xmlrpc php-xmlreader php-xmlwriter \
        php-memcached php-redis php-fpm \
        && rm -rf /var/lib/apt/lists/*
RUN apt update -q && DEBIAN_FRONTEND=noninteractive && { \
    echo mysql-server mysql-server/root_password password 123456; \
    echo mysql-server mysql-server/root_password_again password 123456; \
    } | debconf-set-selections \
    &&  apt install -qy mysql-server && rm -rf /var/lib/apt/lists/*
#todo  config dir replace 
RUN mkdir -p /run/php && mkdir -p /etc/redis && \
rm -rf /etc/nginx/sites-enabled/* && rm -rf /etc/nginx/sites-available/* && \
mv /root/assets/config/nginx/* /etc/nginx/sites-available/ && ln -s /etc/nginx/sites-available/* /etc/nginx/sites-enabled/ && \
/bin/cp -rf /root/assets/config/php-fpm /etc/php/7.0/fpm/pool.d && \
mv /root/assets/config/php/conf.d/* /etc/php/7.0/mods-available/ && \
mv /root/assets/config/redis/* /etc/redis/ && mkdir -p /var/log/redis && \
mkdir -p /var/lib/redis && \
#install server vendor
cd /root/assets/server && tar -zxf redis-stable.tar.gz && \
mv redis-stable /usr/local/redis && cd /usr/local/redis && \
make && make install && cd /root/assets/server && \
tar -xzf zookeeper-3.4.12.tar.gz && mv zookeeper-3.4.12 /usr/local/zookeeper && cd /usr/local/zookeeper/src/c/ && \
./configure && make && make install && rm -rf /usr/local/zookeeper/src && \
mv /root/assets/zoo.cfg /usr/local/zookeeper/conf/ && cd /root/assets/server && \
tar -xjf phantomjs-2.1.1-linux-x86_64.tar.bz2 && mv phantomjs-2.1.1-linux-x86_64 /usr/local/phantomjs && \
ln -s /usr/local/phantomjs/bin/phantomjs /usr/local/bin/phantomjs
#install php-extends
RUN cd /root/assets/php-extend && \
tar -xzf zookeeper-0.6.2.tgz && cd zookeeper-0.6.2 && \
phpize && ./configure && make && make install && cd /etc/php/7.0 && \
ln -s /etc/php/7.0/mods-available/zookeeper.ini cli/conf.d/30-zookeeper.ini && \
ln -s /etc/php/7.0/mods-available/zookeeper.ini fpm/conf.d/30-zookeeper.ini && cd /root/assets/php-extend && \
unzip php-spidermonkey.zip && cd php-spidermonkey && \
phpize && ./configure && make && make install && cd /etc/php/7.0 && \
ln -s /etc/php/7.0/mods-available/spidermonkey.ini cli/conf.d/30-spidermonkey.ini && \
ln -s /etc/php/7.0/mods-available/spidermonkey.ini fpm/conf.d/30-spidermonkey.ini && cd /root/assets/php-extend && \
unzip screw-plus-master.zip && cd screw-plus-master/ && \
/bin/cp -rf /root/assets/config/screw-plus/php_screw_plus.h ./ && \
phpize && ./configure && make && make install && \
cd tools && make && /bin/cp screw /usr/local/bin/ && cd /etc/php/7.0 && \
ln -s /etc/php/7.0/mods-available/screw.ini cli/conf.d/30-screw.ini && \
ln -s /etc/php/7.0/mods-available/screw.ini fpm/conf.d/30-screw.ini
#init mysql data
ADD start.sh /root/
ADD zookeeper.php /root/
RUN chmod +x /root/start.sh && sleep 3 && usermod -d /var/lib/mysql/ mysql && \
service mysql start && sleep 3 &&  mysql -uroot -p123456 < /root/assets/init.sql
#remove anything 
COPY shenjian /data/shenjian/
RUN service mysql stop &&  mkdir -p /data && rm -rf /root/assets
#init pgsql
USER postgres
RUN service postgresql start && psql -c "CREATE USER task_data WITH SUPERUSER PASSWORD '123456';" && createdb -O task_data task_data && \
psql -c "CREATE USER task_queue WITH SUPERUSER PASSWORD '123456';" && createdb -O task_queue task_queue
USER root

VOLUME ["/etc/shenjian", "/var/log/shenjian", "/var/lib/shenjian"]

RUN /bin/cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime #同步时间

EXPOSE 80 22 443
CMD ["/sbin/init"]

• 初始化项目启动脚本:start.sh

#!/bin/bash
#filename:start.sh
#desc:init project and start all server
#author:leeboot
#email:leeboot@163.com

set -e
if [ ! -f /etc/shenjian.init ] ; then
        echo "start init config..."
        #init config dir replace
        if [ ! -d /var/lib/shenjian/mysql ] ; then
                mv /var/lib/mysql /var/lib/shenjian/
        else
                rm -rf /var/lib/mysql
        fi;
        ln -s /var/lib/shenjian/mysql /var/lib/
        if [ ! -d /etc/shenjian/mysql ] ; then
                mv /etc/mysql /etc/shenjian/
        else
                rm -rf /etc/mysql
        fi;
        ln -s /etc/shenjian/mysql /etc/
        if [ ! -d /var/log/shenjian/mysql ] ; then
                mv /var/log/mysql /var/log/shenjian/
        else
                rm -rf /var/log/mysql
        fi;
        ln -s /var/log/shenjian/mysql /var/log/

        sleep 3
        #pgsql
        if [ ! -d /var/lib/shenjian/postgresql ] ; then
                mv /var/lib/postgresql /var/lib/shenjian/
        else
                rm -rf /var/lib/postgresql
        fi;
        ln -s /var/lib/shenjian/postgresql /var/lib/

        if [  ! -d /var/log/shenjian/postgresql ] ; then
                mv /var/log/postgresql /var/log/shenjian/postgresql
        else
                rm -rf /var/log/postgresql
        fi;
        ln -s /var/log/shenjian/postgresql /var/log/

        if [ ! -d /etc/shenjian/postgresql ] ; then
                mv /etc/postgresql /etc/shenjian/
        else
                rm -rf /etc/postgresql
        fi;
        ln -s /etc/shenjian/postgresql /etc/
        #php and nginx
        if [ ! -d /etc/shenjian/php ] ; then
                mv /etc/php /etc/shenjian/
        else
                rm -rf /etc/php
        fi;
        ln -s /etc/shenjian/php /etc/
        if [ ! -d /etc/shenjian/nginx ] ; then
                mv /etc/nginx /etc/shenjian/
        else
                rm -rf /etc/nginx
        fi;
        ln -s /etc/shenjian/nginx /etc/
        if [ ! -d /var/log/shenjian/nginx ] ; then
                mv /var/log/nginx /var/log/shenjian/
        else
                rm -rf /var/log/nginx
        fi;
        ln -s /var/log/shenjian/nginx /var/log/
        #redis
        if [ ! -d /etc/shenjian/redis ] ; then
                mv /etc/redis /etc/shenjian/
        else
                rm -rf /etc/redis
        fi;
        ln -s /etc/shenjian/redis /etc/

        if [ ! -d /var/log/shenjian/redis ] ; then
                mv /var/log/redis /var/log/shenjian/
        else
                rm -rf /var/log/redis
        fi;
        ln -s /var/log/shenjian/redis /var/log/
        if [ ! -d /var/lib/shenjian/redis ] ; then
                mv /var/lib/redis /var/lib/shenjian/
        else
                rm -rf /var/lib/redis
        fi;
        ln -s /var/lib/shenjian/redis /var/lib/
        touch /etc/shenjian.init
        echo "end init config"
fi;


service mysql stop
echo "starting mysql..."
service mysql start

echo "starting postgresql..."
service postgresql start
echo "starting redis..."
cd /etc/redis/
redis-server redis.conf
redis-server sentinel.conf --sentinel

echo "starting zookeeper..."
rm -f /var/lib/zookeeper/zookeeper_server.pid
cd /usr/local/zookeeper/
bin/zkServer.sh start
sleep 3
php /root/zookeeper.php

echo "starting memcached..."
service memcached start

echo "starting nginx..."
chmod 777 /data/shenjian/web/log
php-fpm7.0
nginx
# init service
# this to start you all service
#

• 有些文件就不贴出来了

filename:source.list

deb http://mirrors.aliyun.com/ubuntu/ xenial main
deb-src http://mirrors.aliyun.com/ubuntu/ xenial main

deb http://mirrors.aliyun.com/ubuntu/ xenial-updates main
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-updates main

deb http://mirrors.aliyun.com/ubuntu/ xenial universe
deb-src http://mirrors.aliyun.com/ubuntu/ xenial universe
deb http://mirrors.aliyun.com/ubuntu/ xenial-updates universe
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-updates universe

deb http://mirrors.aliyun.com/ubuntu/ xenial-security main
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-security main
deb http://mirrors.aliyun.com/ubuntu/ xenial-security universe
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-security universe

filename:zookeeper.php
这里需要吐槽一下，java狗，使用zookeeper竟然不在代码中创建，搞得负担到PHP了

<?php
/**
 * Created by PhpStorm.
 * User: leeboot
 * Date: 2018/8/15
 * Time: 15:45
 */

$zc = new Zookeeper();
$zc->connect('localhost:2181');
$aclArray = [
  [
    'perms'  => Zookeeper::PERM_ALL,
    'scheme' => 'world',
    'id'     => 'anyone',
  ]
];
if(!$zc->exists("/crawler")){
  $zc->create("/crawler",null,$aclArray);
  $zc->create("/crawler/apptests",null,$aclArray);
  $zc->create("/crawler/fetchers",null,$aclArray);
  $zc->create("/crawler/workers",null,$aclArray);
}

?>