docker | 搭建 Loki + promtail + gr

涉及组件介绍

• Loki 是 Grafana Labs 团队最新的开源项目，是一个水平可扩展，高可用性，多租户的日志聚合系统
• grafana 美观、强大的可视化监控指标展示工具
• promtail 定时抓取日志文件发送至Loki日志聚合系统进行存储

1. docker 安装 Loki 和 promtail

• docker-compose.yml 文件

version: "3"
services:
  loki:
    image: grafana/loki
    container_name: loki
    restart: always
    ports:
      - "3100:3100"
    volumes:
      - $PWD:/etc/loki
    command: -config.file=/etc/loki/loki-local-config.yaml
  promtail:
    image: grafana/promtail
    container_name: promtail
    restart: always
    volumes:
      - $PWD:/etc/promtail
      - /opt:/opt
    command: 
      -config.file=/etc/promtail/promtail-docker-config.yaml

• 执行安装

docker-compose up -d

2.Loki 和 promtail 配置文件

• Loki 配置文件 loki-local-config.yaml
  可参考官方配置文件: https://github.com/grafana/loki/blob/v1.3.0/docs/configuration/examples.md#complete-local-config
  下面文件是从官方配置文件 Complete Local config 直接复制过来的

auth_enabled: false

server:
  http_listen_port: 3100

ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 5m
  chunk_retain_period: 30s

schema_config:
  configs:
  - from: 2018-04-15
    store: boltdb
    object_store: filesystem
    schema: v9
    index:
      prefix: index_
      period: 168h

storage_config:
  boltdb:
    directory: /tmp/loki/index

  filesystem:
    directory: /tmp/loki/chunks

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h

chunk_store_config:
  max_look_back_period: 0

table_manager:
  chunk_tables_provisioning:
    inactive_read_throughput: 0
    inactive_write_throughput: 0
    provisioned_read_throughput: 0
    provisioned_write_throughput: 0
  index_tables_provisioning:
    inactive_read_throughput: 0
    inactive_write_throughput: 0
    provisioned_read_throughput: 0
    provisioned_write_throughput: 0
  retention_deletes_enabled: false
  retention_period: 0

• promtail 配置文件 promtail-docker-config.yaml
  参考官方文档的: https://github.com/grafana/loki/blob/v1.3.0/docs/clients/promtail/configuration.md#example-docker-config

server: 
  http_listen_port: 0
  grpc_listen_port: 0

positions:
  filename: /etc/promtail/positions.yaml   # 游标记录上一次同步位置
  sync_period: 10s #10秒钟同步一次

clients:
  - url: http://{Loki服务器的ip地址}:3100/loki/api/v1/push

scrape_configs:
- job_name: test-java-log
  static_configs:
  - targets:
      - localhost
    labels:
      job: content-cloud-test
      app: content-cloud-test
      __path__: /opt/content-cloud/webapi/logs/*log   # docker运行时已经把宿主机的目录 /opt/ 映射给了promtail 容器的 /opt/,所以这个地方可以直接访问log文件

root@XXXX:/opt/loki# ll
total 24
drwxr-xr-x  2 root root 4096 Jul  9 16:45 ./
drwxr-xr-x 14 root root 4096 Jul  9 11:14 ../
-rw-r--r--  1 root root  450 Jul  9 16:12 docker-compose.yml
-rw-r--r--  1 root root 1074 Jul  9 14:26 loki-local-config.yaml
-rw-------  1 root root  157 Jul  9 16:45 positions.yaml
-rw-r--r--  1 root root  418 Jul  9 16:00 promtail-docker-config.yaml

3.在grafana 展示日志

1.在grafana 中配置数据源Loki
2.在Explore模块,选择数据源Loki后,就能查看刚才的日志数据了
如果需要在docker中安装grafana请参考 : https://www.jianshu.com/p/87e1ca5b84c9

数据源配置