第八章: kibana画图展示

1.添加索引
[root@elk ~]# vim /etc/filebeat/filebeat.yml 
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/elasticsearch/elasticsearch.log
  multiline.pattern: '^\['
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/elasticsearch/elasticsearch.log
  multiline.pattern: '^\['
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/elasticsearch/elasticsearch.log
  multiline.pattern: '^\['
  multiline.negate: true
  multiline.match: after

output.elasticsearch:
  hosts: ["10.0.0.51:9200"]
  index: "es-%{[beat.version]}-%{+yyyy.MM}"

setup.template.name: "es"
setup.template.pattern: "es_*"
setup.template.enabled: false
setup.template.overwrite: true

2.重启filebeat
systemctl restart filebeat.service

3.上传生成好的日志包并解压追加到日志里
access.tar.gz    #软件包上传
tar xf access.tar.gz 
cat access.log >> /var/log/nginx/access.log

4.查看es-head索引

kibana画图操作

1.- 柱状图 ip url

image.png
image.png
image.png
image.png
image.png
image.png

2.表格 ip url 通用

image.png
image.png
image.png

3.饼图 ua http_code

image.png
image.png
image.png
image.png
image.png

4.词云 ip url 通用

image.png
image.png

5.时间线 pv

image.png
image.png

6.makedown 说明/通知

image.png
image.png

7.dashboard 汇总面板

image.png

第九章:收集tomcat的json日志

1.安装tomcat 
yum install tomcat tomcat-webapps tomcat-admin-webapps tomcat-docs-webapp tomcat-javadoc -y

2.配置tomcat日志格式为json
[root@db-01  ]#  cd /etc/tomcat/
[root@db-01 /etc/tomcat]# sed -n '139p' server.xml 
           pattern="{"clientip":"%h","ClientUser":"%l","authenticated":"%u","AccessTime":"%t","method":"%r","status":"%s","SendBytes":"%b","Query?string":"%q","partner":"%{Referer}i","AgentVersion":"%{User-Agent}i"}"/>

3.启动tomcat
systemctl start tomcat 

4.配置filebeat
[root@db-01 /etc/tomcat]# cat /etc/filebeat/filebeat.yml 
filebeat.inputs:

- type: log
  enabled: true 
  paths:
    - /var/log/tomcat/localhost_access_log.*.txt
  json.keys_under_root: true
  json.overwrite_keys: true
  tags: ["tomcat"]

output.elasticsearch:
  hosts: ["10.0.0.51:9200"]
  index: "tomcat_access-%{[beat.version]}-%{+yyyy.MM}"

setup.template.name: "tomcat"
setup.template.pattern: "tomcat_*"
setup.template.enabled: false
setup.template.overwrite: true
           
5.重启filebeat
systemctl restart filebeat

6.访问tomcat查看是否有数据生成

第十章 收集JAVA日志多行匹配模式

(1).java日志的特点: 

1.报错信息巨多
2.报错信息巨多还是一个事件.不能分开看

(2).匹配思路: 
1.java报错日志特点
正常日志是以[日期]开头的
报错日志行数多,但是不是以[
2.匹配以[开头的行,一直到下一个以[开头的行,中间所有的数据属于一个事件,放在一起发给ES

ilebeat配置多行匹配模式:
[root@db-01 ~]# vim /etc/filebeat/filebeat.yml 
filebeat.inputs:
- type: log
  enabled: true 
  paths:
    - /var/log/elasticsearch/elasticsearch.log
  multiline.pattern: '^\['
  multiline.negate: true
  multiline.match: after

output.elasticsearch:
  hosts: ["10.0.0.51:9200"]
  index: "es-%{[beat.version]}-%{+yyyy.MM}"

setup.template.name: "es"
setup.template.pattern: "es_*"
setup.template.enabled: false
setup.template.overwrite: true