使用Prometheus监控Ceph

在Ceph Luminous之前的版本，可以使用第三方的Prometheus exporterceph_exporter。 Ceph Luminous 12.2.1的mgr中自带了Prometheus插件，内置了 Prometheus ceph exporter，可以使用Ceph mgr内置的exporter作为Prometheus的target。

环境

• ceph dashboard 10.10.1.68
• prometheus 10.10.1.44

方案: 使用ceph mgr prometheus插件

# 启用Ceph的Prometheus插件
ceph mgr module enable prometheus

启用成功后在激活(active)的mgr节点上可以看到

netstat -nltp | grep mgr

root@node1:~# netstat -nltp | grep mgr
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      787/ceph-mgr
tcp        0      0 10.10.1.68:6810         0.0.0.0:*               LISTEN      787/ceph-mgr
tcp        0      0 10.10.1.68:6811         0.0.0.0:*               LISTEN      787/ceph-mgr
tcp6       0      0 :::9283                 :::*                    LISTEN      787/ceph-mgr

其中9283是ceph exporter的监听端口，curl 127.0.0.1:9283/metrics可以获取到metrics。将这个exporter作为Prometheus的target即可

scrape_configs:
  - job_name: 'ceph'
    honor_labels: true
    static_configs:
    - targets: ['10.10.1.68:9283']
      labels:
        instance: Ceph测试集群

接下来安装Prometheus
国内下载

# 下载
wget https://github.com/prometheus/prometheus/releases/download/v2.6.0/prometheus-2.6.0.linux-amd64.tar.gz

# 解压
 tar xf prometheus-2.6.0.linux-amd64.tar.gz

# 把解压出来的文件移动到/usr/local/目录下，并重命名为prometheus
mv prometheus-2.6.0.linux-amd64 /usr/local/prometheus

# 生成启动脚本
$ vim /usr/lib/systemd/system/prometheus.service
# ./prometheus -h 查看帮助
# 添加一下内容
[Unit]
Description=Prometheus: the monitoring system
Documentation=http://prometheus.io/docs/
[Service]
ExecStart=/usr/local/prometheus/prometheus \
        --config.file=/usr/local/prometheus/prometheus.yml \
        --storage.tsdb.path=/var/lib/prometheus \
        --web.console.templates=/usr/local/prometheus/consoles \
        --web.console.libraries=/usr/local/prometheus/console_libraries \
        --web.listen-address=0.0.0.0:9090 --web.external-url=
Restart=always
StartLimitInterval=0
RestartSec=10
[Install]
WantedBy=multi-user.target

# 创建监控数据存储目录
mkdir /var/lib/prometheus

# 启动Prometheus
systemctl daemon-reload
systemctl enable prometheus
systemctl start prometheus

# 查看端口监听状态
netstat -antpu | grep 9090

[root@ceph-admin prometheus]# netstat -antpu | grep 9090
tcp        0      0 127.0.0.1:59856         127.0.0.1:9090          ESTABLISHED 4146/prometheus
tcp6       0      0 :::9090                 :::*                    LISTEN      4146/prometheus
tcp6       0      0 127.0.0.1:9090          127.0.0.1:59856         ESTABLISHED 4146/prometheus
tcp6       0      0 ::1:9090                ::1:58564               ESTABLISHED 4146/prometheus
tcp6       0      0 ::1:58564               ::1:9090                ESTABLISHED 4146/prometheus


# 通过浏览器进行访问
http://ip:9090

# 修改Prometheus配置
vim /usr/local/prometheus/prometheus.yml

# 添加一下内容
scrape_configs:
  - job_name: 'ceph'
    honor_labels: true
    static_configs:
    - targets: ['10.10.1.68:9283']
      labels:
        instance: Ceph测试集群

# 重启Prometheus进程
systemctl restart prometheus

Docker方式部署

# alertmanager
docker run -d -p 9093:9093 -v /usr/local/alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml --name alertmanager docker.io/prom/alertmanager:latest

# prometheus
docker run -d -p 9090:9090 -v /usr/local/promethues/prometheus.yml:/etc/prometheus/prometheus.yml -v /usr/local/promethues/rules:/etc/prometheus/rules --name prometheus docker.io/prom/prometheus