Rancher安装

Rancher是一个开源的容器管理平台，帮助构建企业私有容器服务，相当于KVM里的Openstack。

image.png

直接Docker安装：

$ sudo docker run -d --restart=unless-stopped -p 8080:8080 rancher/server

访问8080端口即可看到界面。
由于众所周知的原因，推荐使用DaoCloud进行Docker安装和镜像加速。
笔者在CentOS 7.2里执行上述命令时遇到一个错误，详情请参考这里。

Rancher认证配置

image.png image.png

啪啪两步搞定。

Rancher主机添加

Rancher提供了很多machine drivers批量添加主机，如下图：

image.png

如果没有可用driver, 就只能选择Custom，看到下图所示：

image.png

在主机执行上图命令即可，See。

image.png

Custom方式批量添加主机

逐台主机登录输入命令也挺头疼，使用expect编写一个脚本减轻部分工作量：

#!/usr/bin/expect
set timeout -1
set ip [lindex $argv 0]
spawn ssh root@$ip
expect {
    "*Are you sure you want to continue connecting (yes/no)?*" { send "yes\r" }
    "*password*" { send "123456\r" }
}
expect {
    "*password*" { send "123456\r" }
}
expect {
    "]# " { send "sudo docker run --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.2 http://123.207.234.119:8080/v1/scripts/EE431DCFC650693D335A:1483142400000:3FSEGIHT9rsY68RViGJGV8sBiBk\r" }
}
send "exit\r"
expect eof
exit

Host配置

在腾讯云申请了6台主机，计划1台master，5台slave。

image.png

Add Host后如下图：

image.png

在一台准备做master的主机add label io.rancher.host.name=master，这个标签在docker compose里用来区分主从机:

image.png

Slave镜像制作

The master-container inside the host can NOT talk to the slave-containers on the other hosts – because containers on each host will be in their own separate network. So they can not communicate.

image.png
Port Mapping: While creating a container, we will be mapping the exposed ports of containers to a host port. So, by talking to host on the mapped port, you will be talking to the actual container.
java.rmi.server.hostname Property: As the containers have their own ip addresses, we need to make Jmeter to communicate via host ip by updating java.rmi.server.hostname. For more info on java rmi properties, check here.

So, 我们需要重新制作一个指定Host IP的slave镜像，这里用环境变量取值实现，dockerfile如下：

FROM malfurionpd/jmeter-base
MAINTAINER adeng <343753091@qq.com>

# Ports to be exposed from the container for JMeter Slaves/Server
EXPOSE 1099 50000

# Application to run on starting the container
ENTRYPOINT $JMETER_HOME/bin/jmeter-server \
                        -Dserver.rmi.localport=50000 \
                        -Dserver_port=1099 \
                        -Djava.rmi.server.hostname=${HOST_IP}

添加应用

Add Stack 就是填写docker-compose.yml和rancher-compose.yml。

image.png

由于slave集群在压测过程中往往由于压力过大出现各种rmi异常导致崩溃，我们把主从分别建两个Stack。

image.png

jmeter-master的docker-compose.yml配置如下：

version: '2'
services:
  master:
    image: malfurionpd/jmeter-master
    stdin_open: true
    network_mode: host
    tty: true
    ports:
    - 60000:60000/tcp
    labels:
      io.rancher.scheduler.affinity:host_label: io.rancher.host.name=master
      io.rancher.container.pull_image: always

jmeter-master的rancher-compose.yml配置如下：

version: '2'
services:
  master:
    scale: 1
    start_on_create: true

jmeter-slaves的docker-compose.yml配置如下：

version: '2'
services:
  slave01:
    image: malfurionpd/jmeter-slave-ip
    environment:
      HOST_IP: 10.0.7.14
    stdin_open: true
    tty: true
    ports:
    - 1099:1099/tcp
    - 50000:50000/tcp
    labels:
      io.rancher.container.pull_image: always
      io.rancher.scheduler.affinity:host_label: host.ip=10.0.7.14
  slave02:
    image: malfurionpd/jmeter-slave-ip
    environment:
      HOST_IP: 10.0.7.7
    stdin_open: true
    tty: true
    ports:
    - 1099:1099/tcp
    - 50000:50000/tcp
    labels:
      io.rancher.container.pull_image: always
      io.rancher.scheduler.affinity:host_label: host.ip=10.0.7.7
  slave03:
    image: malfurionpd/jmeter-slave-ip
    environment:
      HOST_IP: 10.0.7.5
    stdin_open: true
    tty: true
    ports:
    - 1099:1099/tcp
    - 50000:50000/tcp
    labels:
      io.rancher.container.pull_image: always
      io.rancher.scheduler.affinity:host_label: host.ip=10.0.7.5
  slave04:
    image: malfurionpd/jmeter-slave-ip
    environment:
      HOST_IP: 10.0.7.3
    stdin_open: true
    tty: true
    ports:
    - 1099:1099/tcp
    - 50000:50000/tcp
    labels:
      io.rancher.container.pull_image: always
      io.rancher.scheduler.affinity:host_label: host.ip=10.0.7.3
  slave05:
    image: malfurionpd/jmeter-slave-ip
    environment:
      HOST_IP: 10.0.7.11
    stdin_open: true
    tty: true
    ports:
    - 1099:1099/tcp
    - 50000:50000/tcp
    labels:
      io.rancher.container.pull_image: always
      io.rancher.scheduler.affinity:host_label: host.ip=10.0.7.11

jmeter-slaves的rancher-compose.yml配置如下：

version: '2'
services:
  slave01:
    scale: 1
    start_on_create: true
  slave02:
    scale: 1
    start_on_create: true
  slave03:
    scale: 1
    start_on_create: true
  slave04:
    scale: 1
    start_on_create: true
  slave05:
    scale: 1
    start_on_create: true

瞬间集群搭好了：

image.png

愉快的玩耍

登录master，运行一个jmx测试脚本：

malfurionpd$ ssh root@x.x.x.x
[root@VM_7_2_centos ~]# docker ps
CONTAINER ID        IMAGE                            COMMAND                  CREATED             STATUS              PORTS               NAMES
16599e89729a        malfurionpd/jmeter-master        "/bin/bash"              5 minutes ago       Up 5 minutes                            r-jmeter-master-1-0e2ce4cd
2f413226daf1        rancher/net:holder               "/.r/r /rancher-entry"   2 days ago          Up 2 days                               r-ipsec-ipsec-3-9061379e
32164227a861        rancher/dns:v0.15.1              "/rancher-entrypoint."   2 days ago          Up 2 days                               r-network-services-metadata-dns-3-362d1496
15b3f4d6730d        rancher/net:v0.11.2              "/rancher-entrypoint."   2 days ago          Up 2 days                               r-ipsec-ipsec-cni-driver-3-3430dbdd
a024b39bd9c9        rancher/healthcheck:v0.3.1       "/.r/r /rancher-entry"   2 days ago          Up 2 days                               r-healthcheck-healthcheck-3-e5a0a09c
037ff2e87c84        rancher/metadata:v0.9.1          "/rancher-entrypoint."   2 days ago          Up 2 days                               r-network-services-metadata-3-469fa945
71bf613ede11        rancher/network-manager:v0.7.1   "/rancher-entrypoint."   2 days ago          Up 2 days                               r-network-services-network-manager-3-72af28c8
7275306fe9d1        rancher/agent:v1.2.2             "/run.sh run"            2 days ago          Up 2 days                               rancher-agent
[root@VM_7_2_centos ~]# docker attach 16599e89729a
root@VM_7_2_centos:/# cd /jmeter/apache-jmeter-3.2/bin/
root@VM_7_2_centos:/jmeter/apache-jmeter-3.2/bin# wget https://x.x.x/jmetertest/backend-healthcheck.jmx
--2017-06-15 13:33:13--  https://x.x.x/jmetertest/backend-healthcheck.jmx
Resolving x.x.x (x.x.x)... x.x.x.x
Connecting to x.x.x (x.x.x.x)|x.x.x|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6744 (6.6K) [application/octet-stream]
Saving to: 'backend-healthcheck.jmx'

backend-healthcheck.jmx                     100%[========================================================================================>]   6.59K  --.-KB/s    in 0s

2017-06-15 13:33:13 (1.03 GB/s) - 'backend-healthcheck.jmx' saved [6744/6744]

root@VM_7_2_centos:/jmeter/apache-jmeter-3.2/bin# ./jmeter -n -t backend-healthcheck.jmx -Djava.rmi.server.hostname=10.0.7.2 -Dclient.rmi.localport=60000 -R10.0.7.14,10.0.7.7,10.0.7.5,10.0.7.7,10.0.7.11
Creating summariser <summary>
Created the tree successfully using backend-healthcheck.jmx
Configuring remote engine: 10.0.7.14
Configuring remote engine: 10.0.7.7
Configuring remote engine: 10.0.7.5
Configuring remote engine: 10.0.7.7
Configuring remote engine: 10.0.7.11
Starting remote engines
Starting the test @ Thu Jun 15 13:37:46 UTC 2017 (1497533866012)
Remote engines have been started
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary =      4 in 00:00:01 =    4.9/s Avg:   347 Min:   315 Max:   398 Err:     0 (0.00%)
Tidying up remote @ Thu Jun 15 13:37:47 UTC 2017 (1497533867854)
... end of run

关于测试报告的展示，可以看看这里。
Have fun !

参考1