OpenShift Prometheus添加Alert Rule

Prometheus和Openshift

1. prometheus.yml配置中绑定alertmanager服务

......
alerting:
  alertmanagers:
  - scheme: http
    static_configs:
    - targets:
      - "localhost:9093"

2. prometheus.rules设置prometheus告警规则

...
rules:
- alert: TooManyPods
  expr: kuberlet_running_pod_count > 10
  for: 2m
  labels:
    team: node
  annotations:
    summary: "{{$labels.instance}}: has {{$value}} pods"
    description: "{{$labels.instance}} be cateful"

规则：kuberlet_running_pod_count指标持续2分钟超过10，则发出告警。

3. AlertManager中设置告警接收器alertmanager.yml

global:
  smtp_smarthost: 'mail.xx.com:25'
  smtp_from: 'service@xx.com'
  smtp_auth_username: 'service@xx.com'
  smtp_auth_password: 'password'
  smtp_require_tls: false
route:
  group_by: ['alertname']
  receiver: alert-email
receivers:
- name: alert-email
  email_configs:
  - to: 'pan@xx.com'
    send_resolved: true