使用loki+promtail+grafana收集展示docke

介绍

• Grafana 提供 Loki 中捕获的日志行的可视化与搜索平台。
• Loki 是日志记录存储与检索引擎。
• Promtail 做为日志采集器，从 flog 中抓取日志行，并通过网关将它们推送给 Loki。

Promtail 是运行环境的代理（或客户端），它捕获日志行并通过网关将它们推送到 Loki 集群。在典型环境中，生成日志的应用程序和代理一起运行，但位置与 Loki 集群不同。Grafana 提供了一种对 Loki 中存储的日志进行查询并可视化查询结果的方法。

部署

通过docker-compose的方式部署服务

version: "3"

networks:
  loki:

services:
  loki:
    image: grafana/loki:2.9.0
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml
    networks:
      - loki

  promtail:
    image: grafana/promtail:2.9.0
    volumes:
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - /var/run/docker.sock:/var/run/docker.sock
      - /data/softs/grafana_promtail_loki/promtail:/etc/promtail
    command: -config.file=/etc/promtail/promtail-docker-config.yaml
    networks:
      - loki

  grafana:
    environment:
      - GF_PATHS_PROVISIONING=/etc/grafana/provisioning
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
    entrypoint:
      - sh
      - -euc
      - |
        mkdir -p /etc/grafana/provisioning/datasources
        cat <<EOF > /etc/grafana/provisioning/datasources/ds.yaml
        apiVersion: 1
        datasources:
        - name: Loki
          type: loki
          access: proxy
          orgId: 1
          url: http://loki:3100
          basicAuth: false
          isDefault: true
          version: 1
          editable: false
        EOF
        /run.sh
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    networks:
      - loki
      

loki

使用默认配置

auth_enabled: false # 是否启用鉴权

server:
  http_listen_port: 3100 # http访问端口

common:
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks # 记录块存储目录，默认chunks块上的日志数量或到期后，将chunks数据打标签后存储
      rules_directory: /loki/rules # 规则配置目录
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

schema_config:  # 配置索引信息
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_  # 索引前缀
        period: 24h  # 索引时长

ruler:
  alertmanager_url: http://localhost:9093 # 告警通知url

# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
#
# Statistics help us better understand how Loki is used, and they show us performance
# levels for most users. This helps us prioritize features and documentation.
# For more information on what's sent, look at
# https://github.com/grafana/loki/blob/main/pkg/usagestats/stats.go
# Refer to the buildReport method to see what goes into a report.
#
# If you would like to disable reporting, uncomment the following lines:
#analytics:
#  reporting_enabled: false

promtail

1. 默认配置，可以通过挂载/var/log目录获取主机日志

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml # 用于记录每次读取日志文件的索引行数，如：promtail重启后从该配置中恢复日志文件的读取位置

clients:
  - url: http://loki:3100/loki/api/v1/push # 推送日志流到Loki中的api

scrape_configs: # 发现日志文件的位置并从中提取标签
- job_name: system  # 任务名称
  static_configs: # 目录配置
  - targets: # 标签
      - localhost
    labels:
      job: varlogs # 子任务名称，通常以项目命令
      __path__: /var/log/*log # 要读取的日志文件的位置，允许使用通配符/*log或/**/*.log

2. 通过docker
   使用docker_sd_configs提供程序，并仅筛选具有docker标签logging=promtail的docker容器，一旦我们获得这些日志，我们就会重新标记我们的标签以包含容器名称，我们还使用docker标签如log_stream和logging_jobname来为我们的日志添加标签。
   promtail-docker-config.yaml

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://loki:3100/loki/api/v1/push

scrape_configs:
  - job_name: flog_scrape
    docker_sd_configs:
      - host: unix:///var/run/docker.sock
        refresh_interval: 5s
        filters:
          - name: label
            values: ["logging=promtail"]
    relabel_configs:
      - source_labels: ['__meta_docker_container_name']
        regex: '/(.*)'
        target_label: 'container'
      - source_labels: ['__meta_docker_container_log_stream']
        target_label: 'logstream'
      - source_labels: ['__meta_docker_container_label_logging_jobname']
        target_label: 'job'

在容器上添加标签 logging: "promtail"

访问

访问http://ip:3000，点击菜单 Explore，即可查看日志，通过Label filters选择 container，查看各个容器日志