Promtail、Loki、Grafana、Prometheus

Loki: like Prometheus, but for logs.

A Loki-based logging stack consists of 3 components:

• promtail is the agent, responsible for gathering logs and sending them to Loki.
• loki is the main server, responsible for storing logs and processing queries.
• Grafana for querying and displaying the logs.

Loki Explorer

Log browser: {filename="/data/promtail/logs/authservice-ms.log"} |= "MF0A171130770809F5"

Promtail

Promtail is an agent which ships the contents of local logs to a private Grafana Loki instance or Grafana Cloud. It is usually deployed to every machine that has applications needed to be monitored.
It primarily:
• Discovers targets
• Attaches labels to log streams
• Pushes them to the Loki instance.

• Promtail scrape_configs;
  • job_name：在 UI 显示。
  • static_configs：A static_configs allows specifying a list of targets and a common label set for them. It is the canonical way to specify static targets in a scrape configuration.
• Promtail static_configs;
  Promtail 示例

Prometheus

• Template reference：模版参考指南。
• Awesome Prometheus alerts: Collection of alerting rules。
• node 和 npm 安装使用、npm の 常用 packages。

Rules

• Recording rules：记录。
• promtool check rules /path/to/example.rules.yml 检查文件配置语法是否正确

Portainer

• CE 2.11 Portainer architecture

Loki

• 安装 Loki；Loki Documentation；

Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus.

• LogQL: Log Query Language.

LogQL is Loki’s PromQL-inspired query language. Queries act as if they are a distributed grep to aggregate log sources. LogQL uses labels and operators for filtering.

• How to search logs in Loki without worrying about the case?
  {job="varlogs"}|~ "(?i)MF0A1611"：不区分大小写搜索 MF0A1611。
• 2018-03 Loki Design Document；