环境

windows10+elasticsearch7+kibana7+filebeat7

下载

https://www.elastic.co

image.png

https://www.elastic.co/downloads/

image.png

https://www.elastic.co/downloads/beats

image.png

https://www.elastic.co/downloads/beats/filebeat

image.png

https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.6.2-windows-x86_64.zip

配置

文件名：filebeat.yml

#记得开启这个
enabled: true

image.png

启动
https://www.elastic.co/downloads/beats/filebeat

image.png

也可以这样启动

filebeat.exe

image.png

启动成功
data这个目录是记录日志文件的偏移

data这个目录是记录日志文件的偏移

kibana可以看到刚才filebeat传过来的日志生成索引

image.png image.png

filebeat-7.6.2*

image.png image.png image.png image.png

可以查询数据量，看到数据量和文件是一样的。都是14笔数据。

GET /filebeat-7.6.2-2020.04.06-000001/_search

image.png image.png

官网入门帮助

https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html#getting-started

image.png

https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-install.html

image.png

https://www.elastic.co/guide/en/elasticsearch/reference/current/zip-windows.html

安装成windwos服务

image.png

elasticsearch-service.bat install

image.png image.png

elasticsearch-service.bat start

image.png

高级

GET /_cat/indices

DELETE /filebeat-7.6.2-2020.04.12-000001

GET /filebeat-7.6.2-2020.04.12-000001/_search
GET /filebeat-7.6.2-2020.04.12-000001
PUT _ingest/pipeline/filebeat-pipeline
{
  "description" : "格式化日志",
  "processors" : [{
    "grok": {
      "field": "message",
      "patterns": ["%{TIMESTAMP_ISO8601:log_time} \\| %{DATA:log_thread} \\| %{DATA:log_level} \\| %{DATA:log_logger} \\| %{DATA:log_traceId} \\| %{GREEDYDATA:log_msg}"],
      "ignore_failure": true
    },"date": {
      "field": "log_time",
      "formats": ["yyyy-MM-dd HH:mm:ss.SSS"],
      "target_field": "@timestamp",
      "timezone": "Asia/Shanghai"
    },"json": {
      "field": "log_msg"
    }
 }]
} 

image.png

文件名：filebeat.yml

  pipeline: filebeat-pipeline
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
  multiline.negate: true
  multiline.match: after

tags: ["myLog"]

image.png