- 本文安装的是Prometheus-Operator，目前是2.1.0版本（github项目链接）
部署成功效果图
1. 环境介绍
- Docker版本：18.06.1-ce
- Go版本： 1.11.1
- Kubernetes(k8s)版本：1.12.1
- 3台2核内存4G搭载64位centos7系统的云主机
| 主机名 | IP | cpu | 内存 |
|---|---|---|---|
| k8s-master | 192.16.1.215 | 2 | 4G |
| k8s-node1 | 192.16.1.148 | 2 | 4G |
| k8s-node2 | 192.16.1.216 | 2 | 4G |
本人的安装记录如下，具体可参考：
centos7下使用kubeadm安装kubernetes
Centos7下Docker的安装
CentOS7.3安装Go运行和开发环境
在 Kubernetes 集群部署 Helm 与 Tiller server
docker pull jimmysong/kubernetes-helm-tiller:v2.8.1
docker tag jimmysong/kubernetes-helm-tiller:v2.8.1 gcr.io/kubernetes-helm/tiller:v2.8.1
$ wget -qO- https://kubernetes-helm.storage.googleapis.com/helm-v2.8.1-linux-amd64.tar.gz | tar -zx
$ sudo mv linux-amd64/helm /usr/local/bin/
$ kubectl -n kube-system create sa tiller
$ kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
$ helm init --service-account tiller
在 k8s-m1 透过 kubectl 来创建 Ingress Controller 即可：
$ kubectl create ns ingress-nginx
$ wget https://kairen.github.io/files/manual-v1.10/addon/ingress-controller.yml.conf -O ingress-controller.yml
$ sed -i ingress-controller.yml 's/192.16.35.10/172.22.132.10/g'
$ kubectl apply -f ingress-controller.yml.conf
2. 部署Prometheus-Operator
git clone https://github.com/camilb/prometheus-kubernetes.git
cd prometheus-kubernetes
2.1 镜像准备
- PS：如果能科学上网的话就直接忽略，跳到2.2节
2.1.1 方法1 ：直接修改配置文件（推荐）
国内网络下载不了来自k8s.gcr.io的镜像，需要修改镜像源，去docker hub找到了代替的镜像
vim manifests/kube-state-metrics/kube-state-metrics.de.yaml
将k8s.gcr.io/kube-state-metrics:v1.3.1修改成quay.io/coreos/kube-state-metrics:v1.3.1，k8s.gcr.io/addon-resizer:1.7修改成siriuszg/addon-resizer:1.7
2.1.2 方法2 ：提前下载好替代镜像并修改tag
因为不知道pod会在哪个节点上面跑,需要在每个节点做如下操作:
docker pull huwanyang168/kube-state-metrics:v1.3.1
docker tag huwanyang168/kube-state-metrics:v1.3.1 k8s.gcr.io/kube-state-metrics:v1.3.1
docker pull siriuszg/addon-resizer:1.7
docker tag siriuszg/addon-resizer:1.7 k8s.gcr.io/addon-resizer:1.7
2.2 部署
以下不选就是默认为中括号里的选项
./deploy
Check for uncommitted changes
...
...
Enter desired namespace to deploy prometheus [monitoring]:
Creating monitoring namespace.
Error from server (AlreadyExists): namespaces "monitoring" already exists
1) AWS
2) GCP
3) Azure
4) Custom
Please select your cloud provider:4
Deploying on custom providers without persistence
Setting components version
Enter Prometheus Operator version [v0.23.1]:
......
Do you want to set up slack alerts?
Y/N [N]:
Removing all the sed generated files
Deploying Prometheus Operator
...
Waiting for Operator to register custom resource definitions...done!
Deploying ......
Done
2.3 配置集群外也可访问可视化界面
2.3.1 修改 Service Type 类型为 NodePort
由于默认的服务类型都是 ClusterIP，无法集群外部访问,需要将其 Service Type 类型修改为 NodePort。
$ kubectl edit svc grafana -n monitoring
......
spec:
clusterIP: 10.106.93.254
externalTrafficPolicy: Cluster
ports:
- nodePort: 30077
port: 3000
protocol: TCP
targetPort: web
selector:
app: grafana
sessionAffinity: None
type: NodePort # 这里将 ClusterIP 修改为 NodePort
status:
同理,修改 prometheus-k8s、alertmanager-main的Service Type
kubectl edit svc prometheus-k8s -n monitoring
kubectl edit svc alertmanager-main -n monitoring
2.3.2 查看pod所在节点及端口
$ kubectl get svc -n monitoring
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
alertmanager-main NodePort 10.104.133.226 <none> 9093:31254/TCP 28h
alertmanager-operated ClusterIP None <none> 9093/TCP,6783/TCP 28h
grafana NodePort 10.97.217.162 <none> 3000:30015/TCP 28h
kube-state-metrics ClusterIP 10.102.154.120 <none> 8080/TCP 28h
node-exporter ClusterIP None <none> 9100/TCP 28h
prometheus-k8s NodePort 10.96.229.129 <none> 9090:31089/TCP 28h
prometheus-operated ClusterIP None <none> 9090/TCP 28h
prometheus-operator ClusterIP 10.100.13.182 <none> 8080/TCP 28h
$ kubectl get pods -n monitoring -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
alertmanager-main-0 2/2 Running 0 28h 172.17.0.10 weiyi-workerb.novalocal <none>
alertmanager-main-1 2/2 Running 0 28h 172.17.0.12 swarmworkera <none>
alertmanager-main-2 2/2 Running 0 28h 172.17.0.6 weiyi-docker-master.novalocal <none>
grafana-69dc97645f-wjcz6 1/1 Running 0 28h 172.17.0.11 weiyi-workerb.novalocal <none>
kube-state-metrics-5bc5fbb5c6-5t97x 2/2 Running 0 28h 172.17.0.14 swarmworkera <none>
node-exporter-7nvjg 1/1 Running 0 28h 192.16.1.216 weiyi-workerb.novalocal <none>
node-exporter-8vlcp 1/1 Running 0 28h 192.16.1.148 swarmworkera <none>
node-exporter-wplqf 1/1 Running 0 28h 192.16.1.215 weiyi-docker-master.novalocal <none>
prometheus-k8s-0 3/3 Running 1 28h 172.17.0.5 weiyi-docker-master.novalocal <none>
prometheus-k8s-1 3/3 Running 1 28h 172.17.0.13 swarmworkera <none>
prometheus-operator-54fd688cd8-658lp 1/1 Running 0 28h 172.17.0.4 weiyi-docker-master.novalocal <none>
根据如上所示得知:
grafana界面: 192.16.1.216:30015
prometheus界面:192.16.1.148:31089和192.16.1.215:31089
alertmanager界面:192.16.1.148:31089,192.16.1.215:31089和192.16.1.216:31089
2.3.3 查看可视化界面
grafana界面
promethus界面
alertmanager界面
2.4 查看部署情况
$ kubectl -n monitoring get pods
NAME READY STATUS RESTARTS AGE
alertmanager-main-0 2/2 Running 0 25h
alertmanager-main-1 2/2 Running 0 25h
alertmanager-main-2 2/2 Running 0 25h
grafana-69dc97645f-wjcz6 1/1 Running 0 25h
kube-state-metrics-5bc5fbb5c6-5t97x 2/2 Running 0 25h
node-exporter-7nvjg 1/1 Running 0 25h
node-exporter-8vlcp 1/1 Running 0 25h
node-exporter-wplqf 1/1 Running 0 25h
prometheus-k8s-0 3/3 Running 1 25h
prometheus-k8s-1 3/3 Running 1 25h
prometheus-operator-54fd688cd8-658lp 1/1 Running 0 25h
$ kubectl get svc -n monitoring
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
alertmanager-main NodePort 10.104.133.226 <none> 9093:31254/TCP 25h
alertmanager-operated ClusterIP None <none> 9093/TCP,6783/TCP 25h
grafana NodePort 10.97.217.162 <none> 3000:30015/TCP 25h
kube-state-metrics ClusterIP 10.102.154.120 <none> 8080/TCP 25h
node-exporter ClusterIP None <none> 9100/TCP 25h
prometheus-k8s NodePort 10.96.229.129 <none> 9090:31089/TCP 25h
prometheus-operated ClusterIP None <none> 9090/TCP 25h
prometheus-operator ClusterIP 10.100.13.182 <none> 8080/TCP 25h
$ kubectl -n monitoring get crd
NAME CREATED AT
alertmanagers.monitoring.coreos.com 2018-10-30T07:11:12Z
prometheuses.monitoring.coreos.com 2018-10-30T07:11:12Z
prometheusrules.monitoring.coreos.com 2018-10-30T07:11:12Z
servicemonitors.monitoring.coreos.com 2018-10-30T07:11:12Z
$ kubectl -n monitoring get servicemonitors
NAME AGE
alertmanager 1d
kube-apiserver 1d
kube-controller-manager 1d
kube-dns 1d
kube-scheduler 1d
kube-state-metrics 1d
kubelet 1d
node-exporter 1d
prometheus 1d
prometheus-operator 1d
参考文献
Prometheus Operator 介绍与安装
部署 Prometheus Operator 监控 Kubernetes 集群
Kubernetes Nginx Ingress 教程
网友评论