kolla

2018-05-13

源码解析

• 需要了解Dockerfile相关知识

https://hujb2000.gitbooks.io/docker-flow-evolution/content/cn/basis/dockerfiledetail.html
http://seanlook.com/2014/11/17/dockerfile-introduction/

• 源码结构如下

├── bindep.txt
├── contrib 该目录下存放非镜像制作的模板，这些文件需要在对应的doc文件夹下存在描述文档
│   └── template-override
├── deliverables.yaml
├── doc 文档描述文件
│   ├── requirements.txt
│   └── source
│       └── admin
├── docker 构建docker镜像的j2模板
│   ├── nova
│   │   └── nova-api
│   │      ├── Dockerfile.j2
│   │      └── extend_start.sh
├── etc 包含小部分/etc目录下的配置文件用于AIO测试
│   ├── kolla kolla-build.conf文件所在文件夹
│   └── oslo-config-generator 定义kolla-build.conf源文件所在位置
├── HACKING.rst
├── kolla
│   ├── cmd 调用image下的build.py并将结果返回
│   ├── common
│       ├── config.py 定义发行版字典，标准库地址，source源码地址，用户uid等
│       ├── utils.py 打印日志精简镜像
│   ├── exception.py 自定义忽略的异常
│   ├── hacking
│   ├── image build镜像代码所在
│       ├── build.py
│       ├── 1.定义状态、定义不同操作系统+安装方式所不能安装的组件镜像
│       ├── 2.定义Image类，属性及copy方法
│       ├── 3. 定义PushTask及PushQueueTask类，将镜像推到docker仓库
│       ├──
│   ├── opts.py 返回配置文件内容
│   ├── template
│       ├── filter.py 未发现被加载调用
│       ├── methods.py 安装debian-based的软件包 repo地址等,未发现加载调用
│   ├── tests 创建镜像mock测试相关代码，一些pip包需要安装
│   ├── version.py 返回kolla版本信息
├── kolla.egg-info kolla项目文件打包信息
│   ├── dependency_links.txt 空
│   ├── entry_points.txt
│   ├── not-zip-safe
│   ├── pbr.json
│   ├── PKG-INFO 包信息
│   ├── requires.txt 依赖包的版本信息
│   ├── SOURCES.txt 资源依赖列表
│   └── top_level.txt kolla tox.ini会加载该文件
├── LICENSE
├── lower-constraints.txt
├── README.rst 项目简介
├── releasenotes 版本信息描述，
│   ├── notes 所有组件的信息描述
│   ├── source 是各个版本稳定代码下载位置
│   └── upgrage-to-mariadb-10.1.20-for-redhat-family-93df8bf63da25659.yaml
├── requirements.txt
├── roles ansible的roles文件夹
│   ├── collect-collectd 安装collect服务，更改配置文件并收集主机信息
│   └── collectd
├── setup.cfg 入口文件
├── setup.py
├── specs
│   ├── ansible-multi.rst
│   ├── containerize-openstack.rst
│   ├── ha.svg
│   ├── high-availability.rst
│   ├── kolla-ceph-bluestore.rst
│   ├── kubernetes-deployment.rst
│   ├── logging-with-heka.rst
│   ├── logging-with-heka.svg
│   ├── README.rst
│   └── template.rst
├── test-requirements.txt
├── tests mock测试代码
│   ├── clients.py
│   ├── __init__.py
│   ├── playbooks
│   ├── setup_gate.sh -> ../tools/setup_gate.sh
│   ├── templates
│   ├── test_build.py
│   ├── test_keystone.py
│   └── test_set_config.py
├── tools kolla工具文件夹
│   ├── build.py -> ../kolla/cmd/build.py 软连接文件
│   ├── cleanup-images 将lable为kolla_version，none的镜像全部删除
│   ├── deploy_aio.sh 调用ansible命令部署all in one环境，与手工过程一致可参考该脚本包装过程
│   ├── diag 全文只有kolla.egg-info/SOURCES.txt中标记过并未有调用
│   ├── dump_info.sh 获取节点的信息如内存数据等，单元测试时使用
│   ├── gate_run.sh 单元测试调用setup_gate方法,被tools/publisher和tests/playbooks/run.yaml调用
│   ├── loc
│   ├── pre-commit-hook 格式检查
│   ├── publisher.sh
│   ├── run-bashate.sh
│   ├── setup_Debian.sh
│   ├── setup_gate.sh 创建一个4000：5000映射的docker仓库
│   ├── setup_RedHat.sh
│   ├── start-registry 创建一个端口5000映射的docker仓库
│   ├── validate-all-dockerfiles.sh 调用validate-maintainer.sh、validate-install-command.sh、validate-indentation.sh脚本进行dockerfiles.j2的验证
│   ├── validate-all-file.py 检查定义后缀名的文件内容是否正确
│   ├── validate-all-yaml.sh 调用validate-yaml.py文件
│   ├── validate-docker-execute.sh 验证当前用户是否能够使用docker命令
│   ├── validate-indentation.sh
│   ├── validate-install-command.sh 验证apt-get或者yum安装命令是否可用
│   ├── validate-maintainer.sh 检查维护者是不是参数中的值
│   ├── validate-yaml.py 通过将yaml转换成python对象来检查yaml文件正确性
│   └── version-check.py
└── tox.ini tox配置文件

kolla提供openstack image包括

• Almanach <https://almanach.readthedocs.io/>__
• Aodh <https://docs.openstack.org/aodh/latest/>__
• Barbican <https://docs.openstack.org/barbican/latest/>__
• Bifrost <https://docs.openstack.org/bifrost/latest/>__
• Blazar <https://blazar.readthedocs.io/en/latest/>__
• Ceilometer <https://docs.openstack.org/ceilometer/latest/>__
• Cinder <https://docs.openstack.org/cinder/latest/>__
• CloudKitty <https://docs.openstack.org/cloudkitty/latest/>__
• Congress <https://docs.openstack.org/congress/latest/>__
• Designate <https://docs.openstack.org/designate/latest/>__
• Dragonflow <https://docs.openstack.org/dragonflow/latest/>__
• EC2-API <https://wiki.openstack.org/wiki/EC2API>__
• Freezer <https://docs.openstack.org/freezer/latest/>__
• Glance <https://docs.openstack.org/glance/latest/>__
• Heat <https://docs.openstack.org/heat/latest/>__
• Horizon <https://docs.openstack.org/horizon/latest/>__
• Ironic <https://docs.openstack.org/ironic/latest/>__
• Karbor <https://docs.openstack.org/karbor/latest/>__
• Keystone <https://docs.openstack.org/keystone/latest/>__
• Kuryr <https://docs.openstack.org/kuryr/latest/>__
• Magnum <https://docs.openstack.org/magnum/latest/>__
• Manila <https://docs.openstack.org/manila/latest/>__
• Mistral <https://docs.openstack.org/mistral/latest/>__
• Monasca <https://wiki.openstack.org/wiki/Monasca>__
• Murano <https://docs.openstack.org/murano/latest/>__
• Neutron <https://docs.openstack.org/neutron/latest/>__
• Nova <https://docs.openstack.org/nova/latest/>__
• Octavia <https://docs.openstack.org/octavia/latest/>__
• Panko <https://docs.openstack.org/panko/latest/>__
• Rally <https://docs.openstack.org/rally/latest/>__
• Sahara <https://docs.openstack.org/sahara/latest/>__
• Searchlight <https://docs.openstack.org/searchlight/latest/>__
• Senlin <https://docs.openstack.org/senlin/latest/>__
• Solum <https://docs.openstack.org/solum/latest/>__
• Swift <https://docs.openstack.org/swift/latest/>__
• Tacker <https://docs.openstack.org/tacker/latest/>__
• Tempest <https://docs.openstack.org/tempest/latest/>__
• Trove <https://docs.openstack.org/trove/latest/>__
• Vitrage <https://docs.openstack.org/vitrage/latest/>__
• Vmtp <https://vmtp.readthedocs.io/en/latest/>__
• Watcher <https://docs.openstack.org/watcher/latest/>__
• Zaqar <https://docs.openstack.org/zaqar/latest/>__
• Zun <https://docs.openstack.org/zun/latest/>_

kolla提供的基础镜像包括

• Ceph <https://ceph.com/>__ implementation for Cinder, Glance and Nova
• Chrony <https://chrony.tuxfamily.org/>__ a versatile implementation
  of the Network Time Protocol (NTP).
• Collectd <https://collectd.org>,
  InfluxDB <https://influxdata.com/time-series-platform/influxdb/>, and
  Grafana <https://grafana.org>__ for performance monitoring.
• Elasticsearch <https://www.elastic.co/de/products/elasticsearch>__ and
  Kibana <https://www.elastic.co/products/kibana>__ to search, analyze,
  and visualize log messages.
• Etcd <https://coreos.com/etcd/>__ a distributed key value store that provides
  a reliable way to store data across a cluster of machines.
• Fluentd <https://www.fluentd.org/>__ as an open source data collector
  for unified logging layer.
• Gnocchi <http://gnocchi.xyz/>__ A time-series storage database.
• HAProxy <https://www.haproxy.org/>__ and
  Keepalived <http://www.keepalived.org/>__ for high availability of services
  and their endpoints.
• Kafka <https://kafka.apache.org/documentation/>__ A distributed streaming
  platform.
• MariaDB and Galera Cluster <https://mariadb.com/kb/en/library/galera-cluster/>__
  for highly available MySQL databases.
• Memcached <https://www.memcached.org/>__ a distributed memory object caching system.
• MongoDB <https://www.mongodb.org/>__ as a database back end for Panko.
• Open vSwitch <http://openvswitch.org/>__ and Linuxbridge back ends for Neutron.
• Linux ptp <http://linuxptp.sourceforge.net/>__ an implementation of the Precision
  Time Protocol (PTP) according to IEEE standard 1588 for Linux.
• Qdrouterd <https://qpid.apache.org/components/dispatch-router/index.html>__ as a
  direct messaging back end for communication between services.
• RabbitMQ <https://www.rabbitmq.com/>__ as a broker messaging back end for
  communication between services.
• Telegraf <https://www.docs.influxdata.com/telegraf/>__ as a plugin-driven server
  agent for collecting & reporting metrics.
• ZooKeeper <https://zookeeper.apache.org/>__ as a centralized service for maintaining
  configuration information, naming, providing distributed synchronization, and providing
  group services

kolla的目标是提供一个生产级别的容器化的云平台部署工具

第一章 Kolla的容器镜像制作

1. 确保Kolla已经安装
2. 配置好kolla-build命令或者使用本地tools/build.py脚本进行操作

生成kolla-build.conf文件

• 安装单元测试工具tox,并生成kolla-build.conf文件

pip install tox
cd kolla/
tox -e genconfig

• building kolla image

kolla-build 或者 python tools/build.py

• build单独的组件

kolla-build keystone

• 同时build多个组件

kolla-build keystone nova

从source来build组件

kolla-build -t source

• etc/kolla/kolla-build.conf配置文件需要修改如下

[glance-base]
type = url
location = http://tarballs.openstack.org/glance/glance-master.tar.gz

[keystone-base]
type = git
location = https://git.openstack.org/openstack/keystone
reference = stable/mitaka

[heat-base]
type = local
location = /home/kolla/src/heat

[ironic-base]
type = local
location = /tmp/ironic.tar.gz

OpenDaylight Build (未研究)

OVS-DPDK Source build(未研究)

测试方法(未研究)

• 由于网络原因，修改 ./kolla/docker/base/Dockerfile.j2:75，手工导入key

        'https://packages.elastic.co/GPG-KEY-elasticsearch',
        'https://repos.influxdata.com/influxdb.key',
        'https://packagecloud.io/gpg.key',
        'https://grafanarel.s3.amazonaws.com/RPM-GPG-KEY-grafana',
        'https://packages.treasuredata.com/GPG-KEY-td-agent'

• 修改134行

        'centos-release-ceph-luminous',
        'centos-release-opstools',
        'centos-release-qemu-ev'

• 修改120行

        'epel-release',
        'yum-plugin-priorities'

• 修改124行手工导入以下KEY

        '/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-OpsTools',
        '/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage',
        '/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Virtualization'

• 修改136行，手工执行命令没有需要安全设置的package

    yum -y update --security --sec-severity=Important --sec-severity=Critical \
    && yum clean all

• 修改207行,手工安装

        'iproute',
        'iscsi-initiator-utils',
        'lvm2',
        'python',
        'scsi-target-utils',
        'socat',
        'sudo',
        'which'

• 修改kolla/docker/fluentd/Dockerfile.j2 +19手工安装以下软件

        'fluentd',
        'hostname',
        'rubygem-fluent-plugin-elasticsearch',
        'rubygem-fluent-plugin-grok-parser',
        'rubygem-fluent-plugin-kubernetes_metadata_filter',
        'rubygem-fluent-plugin-rewrite-tag-filter',
        'rubygem-fluent-plugin-secure-forward'

• 制作nova image时候 修改kolla/docker/nova/nova-libvirt/Dockerfile.j2 +19
  包改为手动安装

        'ceph-common',
        'cyrus-sasl-scram',
        'libguestfs',
        'libvirt-client',
        'libvirt-daemon',
        'libvirt-daemon-config-nwfilter',
        'libvirt-daemon-driver-lxc',
        'libvirt-daemon-driver-nwfilter',
        'openvswitch',
        'qemu-kvm'