elk 3：springboot logback采集日志到log

作者: _Rondo | 来源:发表于2020-10-22 09:38 被阅读0次

elk 3：springboot logback采集日志到log
SpringBoot —— 整合Logback，输出日志到文件
使用Log4j日志处理
SpringBoot系列之集成logback实现日志打印(篇二)
SpringBoot使用LogBack日志组件
springboot之配置日志
SpringBoot 篇 - 配置使用Logback进行日志记录
SpringBoot整合logback日志
SpringBoot笔记（六）日志
Spring Boot 默认日志配置

一、前言

看了下马哥的内部资料，关于es 日志采集的部署方案，感觉可以直接 logback 输出到 logstash，再到es

二、安装logstash

没有虚拟机，依然安装的是windows版本，解压后config下新建logstash.conf，添加默认输出内容

input {
  beats {
    port => 5044
  }
}
output {
  stdout { codec => rubydebug }
}

然后bin下新建start.bat，添加以下内容，双击启动

./logstash.bat -f ../config/logstash.conf

三、springboot 集成 logback

添加pom依赖

<dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
        </dependency>

        <!-- https://mvnrepository.com/artifact/com.alibaba/fastjson -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.73</version>
        </dependency>

        <!--logStash-->
        <dependency>
            <groupId>net.logstash.logback</groupId>
            <artifactId>logstash-logback-encoder</artifactId>
            <version>6.3</version>
        </dependency>

更改build

<!-- 资源文件配置 -->
        <resources>
            <resource>
                <directory>src/main/java</directory>
                <excludes>
                    <exclude>**/*.java</exclude>
                </excludes>
            </resource>
            <resource>
                <directory>src/main/resources</directory>
                <includes>
                    <include>*.yml</include>
                    <include>*.xml</include>
                </includes>
            </resource>
        </resources>

resources下新建logback文件

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/base.xml" />
    <appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <destination>127.0.0.1:9999</destination>
        <includeCallerData>true</includeCallerData>

        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <includeCallerData>true</includeCallerData>
        </encoder>
    </appender>
    <logger name="com.example.logbacklogstash" level="debug" additivity="false">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="stash" />
        <appender-ref ref="FILE"/>
    </logger>
    <logger name="root" level="warn" additivity="false">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="stash" />
        <appender-ref ref="FILE"/>
    </logger>
</configuration>

更改spring 配置文件

spring:
  elasticsearch:
    rest:
      uris: http://127.0.0.1:9200

logging:
  config: classpath:logback-spring.xml

添加环绕通知切面

package com.example.logbacklogstash.aspect;

import com.alibaba.fastjson.JSON;
import lombok.extern.slf4j.Slf4j;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.springframework.stereotype.Component;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;
import org.springframework.web.multipart.MultipartFile;

import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;


@Slf4j
@Aspect
@Component
public class SysAccessLog {


    @Pointcut("execution (* com.example.logbacklogstash..*Controller.*(..))")
    public void SysAccessLog() {
    }

    //环绕通知,环绕增强，相当于MethodInterceptor
    @Around(value = "SysAccessLog()")
    public Object doArround(ProceedingJoinPoint pjp) throws Exception {

        long stime = System.currentTimeMillis();

        try {
            // 接收到请求，记录请求内容
            ServletRequestAttributes attributes = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
            HttpServletRequest req = attributes.getRequest();
            // 记录下请求内容
            log.info("[HTTP <<<] ");
            log.info("[HTTP_URL] : {}" , req.getRequestURL().toString());
            log.info("[HTTP_METHOD] : {}" ,  req.getMethod());
            log.info("[HTTP_ACTION] : {}", pjp.getSignature().getDeclaringTypeName()+ "." + pjp.getSignature().getName());
            Object[] args = getMethodArg(pjp.getArgs());
            log.info("[HTTP_PARAMS] : {}", JSON.toJSONString(args[0]));
            log.info("[REMOTE_IP] : {}" ,  req.getRemoteAddr());

            //运行方法
            Object o = pjp.proceed();
            log.info("[HTTP_RESPONSE]: {}",JSON.toJSONString(o));
            return o;
        } catch (Throwable throwable) {
            throw new Exception(throwable.getMessage(), throwable);
        } finally {
            long etime = System.currentTimeMillis();
            log.info("[HTTP_TIME] : {}",  (etime - stime)+"ms");
        }
    }


    private Object[] getMethodArg(Object[] args) {
        Object[] arguments = new Object[args.length];
        for (int i = 0; i < args.length; i++) {
            if (args[i] instanceof ServletRequest || args[i] instanceof ServletResponse || args[i] instanceof MultipartFile) {
                //ServletRequest不能序列化，从入参里排除，否则报异常：java.lang.IllegalStateException: It is illegal to call this method if the current request is not in asynchronous mode (i.e. isAsyncStarted() returns false)
                //ServletResponse不能序列化 从入参里排除，否则报异常：java.lang.IllegalStateException: getOutputStream() has already been called for this response
                continue;
            }
            arguments[i] = args[i];
        }
        return arguments;
    }

}

添加通用结果

package com.example.logbacklogstash.base;

import lombok.Data;

/**
 * 通用结果
 * @author wenx
 * @date 2020-09-30
 */
@Data
public class Result {

    private String code;

    private String msg;
//    private Object data;

    public Result() {
    }

    public Result(String code, String msg) {
        this.code = code;
        this.msg = msg;
    }

    public Result error(){
        return this.render(ResultEnum.error.getCode(),"error");
    }

    public Result error(String msg){
        return this.render(ResultEnum.error.getCode(),msg);
    }

    public Result fail(String msg){
        return this.render(ResultEnum.fail.getCode(),msg);
    }

    public Result fail(){
        return this.render(ResultEnum.fail.getCode(),"fail");
    }

    public Result fail(Object data){
        return this.render(ResultEnum.fail.getCode(),"fail",data);
    }

    public Result success(){
        return this.render(ResultEnum.success.getCode(),"success");
    }
    public Result success(Object data){
        return this.render(ResultEnum.success.getCode(),"success",data);
    }
    public Result success(String msg){
       return this.render(ResultEnum.success.getCode(),msg);
    }

    public Result render(String code, String msg){
        this.code = code;
        this.msg = msg;
        return this;
    }

    public Result render(String ret, String msg, Object data){
        this.code = ret;
        this.msg = msg;
//        this.data = data;
        return this;
    }
}

通用结果枚举

package com.example.logbacklogstash.base;

import lombok.Getter;

/**
 * 返回结果enum
 * @author wenx
 * @date 2020-09-30
 */
@Getter
public enum ResultEnum {
    //结果
    success("1"),fail("0"),error("2");

    private String code;

    ResultEnum(String code) {
        this.code = code;
    }

}

添加一个测试controller

package com.example.logbacklogstash.controller;


import com.example.logbacklogstash.base.Result;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

/**
 * 测试controller 拟定接口模板
 * @author wenx
 * @date 2020-07-16
 */
@Slf4j
@RestController
public class TestController {

    @GetMapping("/api/test")
    public Result testForm(@RequestParam(value = "param",required = false,defaultValue = "")String param){
       log.debug("param:{}",param);
       return new Result().success();
    }

}

最后需要修改刚才的logstash配置然后重启，这里有个误区，之前以为tcp里的port是logstash的端口，后来才发现是交互信息的channel，应该是和socket client一样。

# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
    tcp {
        ##host:port就是上面appender中的 destination，这里其实把logstash作为服务，开启9250端口接收logback发出的消息
        host => "127.0.0.1"
        port => "9999"
        mode => "server"
        tags => ["tags"]
        codec => json_lines
    }
  beats {
    port => 5044
  }
}
filter{
    date{
        match => ["timestamp","dd-MMM-yyyy:HH:mm:ss Z"]
    }
}

output {
  stdout { codec => rubydebug }
  elasticsearch {
    hosts => "localhost:9200" 
    index => "logstash-%{+YYYY.MM.dd}"
  }
  file {
       path => "/data/ELK/logstash/%{+YYYY.MM.dd}-out.txt"
       codec => line 
  }
}

-end-

elk 3：springboot logback采集日志到log
一、前言看了下马哥的内部资料，关于es 日志采集的部署方案，感觉可以直接 logback 输出到 logstas...
SpringBoot —— 整合Logback，输出日志到文件
前言 SpringBoot系列Demo代码，整合logback，设置输出日志。一、logback日志组件 log...
使用Log4j日志处理
Springboot日志默认使用的是logback，本文将介绍将springboot项目日志修改为log4j。首...
SpringBoot系列之集成logback实现日志打印(篇二)
SpringBoot系列之集成logback实现日志打印(篇二) 基于上篇博客SpringBoot系列之集成log...
SpringBoot使用LogBack日志组件
一、引言 SpringBoot内部集成了LogBack日志依赖，SpringBoot默认使用LogBack记录日志...
springboot之配置日志
springboot默认启动的logback日志分析 springboot默认集成了logback日志。说明：2...
SpringBoot 篇 - 配置使用Logback进行日志记录
SpringBoot框架内部集成了LogBack日志，默认使用LogBack的base.xml配置内容来输出日志到...
SpringBoot整合logback日志
SpringBoot内部集成了logback日志依赖，默认使用logback记录日志信息。一、logback介绍...
SpringBoot笔记（六）日志
SpringBoot默认使用Logback作为日志记录。同时也对java.util包的logging和Log4j...
Spring Boot 默认日志配置
springboot 默认日志配置 SpringBoot 日志配置默认采用LogBack作为日志输出！日志格式...