SpringBoot使用Graylog日志收集

作者: dalaoyang | 来源:发表于2019-04-22 16:59 被阅读121次

    本文介绍SpringBoot如何使用Graylog日志收集。

    1.Graylog介绍

    Graylog是一个生产级别的日志收集系统,集成Mongo和Elasticsearch进行日志收集。其中Mongo用于存储Graylog的元数据信息和配置信息,ElasticSearch用于存储数据。

    架构图如下:

    image

    生产环境配置图如下:

    image

    2.安装Graylog

    在官方文档上推荐了很多种安装的方式,这里以docker-compose的方式为例,进行安装Graylog,mongo,elasticsearch。

    docker-compose.yml内容如下(这里是在官网的基础上改了一下):

    version: '2'
    services:
      # MongoDB: https://hub.docker.com/_/mongo/
      mongodb:
        image: mongo:3
      # Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/6.6/docker.html
      elasticsearch:
        image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.6.1
        environment:
          - http.host=0.0.0.0
          - transport.host=localhost
          - network.host=0.0.0.0
          - "ES_JAVA_OPTS=-Xms256m -Xmx256m"
        ulimits:
          memlock:
            soft: -1
            hard: -1
        mem_limit: 512m
      # Graylog: https://hub.docker.com/r/graylog/graylog/
      graylog:
        image: graylog/graylog:3.0
        environment:
          # CHANGE ME (must be at least 16 characters)!
          - GRAYLOG_PASSWORD_SECRET=somepasswordpepper
          # Password: admin
          - GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
          - GRAYLOG_HTTP_EXTERNAL_URI=http://106.13.35.42:9000/
        links:
          - mongodb:mongo
          - elasticsearch
        depends_on:
          - mongodb
          - elasticsearch
        ports:
          # Graylog web interface and REST API
          - 9000:9000
          # Syslog TCP
          - 1514:1514
          # Syslog UDP
          - 1514:1514/udp
          # GELF TCP
          - 12201:12201
          # GELF UDP
          - 12201:12201/udp
    

    其中106.13.35.42是我的外网ip,本地服务使用127.0.0.1即可。

    其他方式可以查看官方文档,https://docs.graylog.org/en/3.0/pages/installation.html

    3.配置Graylog

    在浏览器访问http://ip:9000,如图:

    image

    这里默认用户名密码都是admin,进入后如图所示。

    image

    选择System按钮中的input,录入一个输入源,如图

    image

    这里以GELF UDP为例,在图中位置选择GELF UDP,选择完成后点击Launch new input,如图

    image

    在Node处选择自己安装的,剩下的就根据需要填写即可,如图

    image

    保存完成后如图,到这里就已经配置完成了。

    image

    4.SpringBoot日志输出到Graylog

    这里分别举例Logback日志和Log4j2日志。

    4.1 Logback日志

    这里使用的logback-gelf向Graylog输出日志,在github上有对logback-gelf的详细使用介绍,这里只是简单举例。Github地址:https://github.com/osiegmar/logback-gelf

    新建项目,加入logback-gelf依赖,pom文件如下:

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <parent>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-parent</artifactId>
            <version>2.1.4.RELEASE</version>
            <relativePath/> <!-- lookup parent from repository -->
        </parent>
        <groupId>com.dalaoyang</groupId>
        <artifactId>springboot2_graylog</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <name>springboot2_graylog</name>
        <description>springboot2_graylog</description>
    
        <properties>
            <java.version>1.8</java.version>
        </properties>
    
        <dependencies>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-test</artifactId>
                <scope>test</scope>
            </dependency>
    
            <dependency>
                <groupId>de.siegmar</groupId>
                <artifactId>logback-gelf</artifactId>
                <version>2.0.0</version>
            </dependency>
        </dependencies>
    
        <build>
            <plugins>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                </plugin>
            </plugins>
        </build>
    
    </project>
    
    

    加入logback日志配置,新建logback-spring.xml,内容如下:

    <configuration>
    
        <conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter" />
        <conversionRule conversionWord="wex" converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter" />
        <conversionRule conversionWord="wEx" converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter" />
        <property name="CONSOLE_LOG_PATTERN" value="${CONSOLE_LOG_PATTERN:-%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>
    
        <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
            <encoder>
                <pattern>${CONSOLE_LOG_PATTERN}</pattern>
                <charset>UTF-8</charset>
            </encoder>
        </appender>
    
        <appender name="GELF" class="de.siegmar.logbackgelf.GelfUdpAppender">
            <graylogHost>106.13.35.42</graylogHost>
            <graylogPort>12201</graylogPort>
        </appender>
    
        <!-- 控制台输出日志级别 -->
        <root level="info">
            <appender-ref ref="GELF" />
            <appender-ref ref="STDOUT" />
        </root>
    
    
    </configuration>
    
    

    启动项目,当前项目端口是8081,查看Graylog控制台如图:

    image

    4.2 Log4j2日志

    log4j2日志使用的是log4j2-gelf依赖,github上面也有对应的介绍,pom文件如下:

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <parent>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-parent</artifactId>
            <version>2.1.4.RELEASE</version>
            <relativePath/> <!-- lookup parent from repository -->
        </parent>
        <groupId>com.dalaoyang</groupId>
        <artifactId>springboot2_graylog_log4j</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <name>springboot2_graylog_log4j</name>
        <description>springboot2_graylog_log4j</description>
    
        <properties>
            <java.version>1.8</java.version>
        </properties>
    
        <dependencies>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-test</artifactId>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter</artifactId>
                <exclusions>
                    <exclusion>
                        <artifactId>spring-boot-starter-logging</artifactId>
                        <groupId>org.springframework.boot</groupId>
                    </exclusion>
                </exclusions>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-log4j2</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.graylog2.log4j2</groupId>
                <artifactId>log4j2-gelf</artifactId>
                <version>1.3.1</version>
            </dependency>
    
        </dependencies>
    
        <build>
            <plugins>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                </plugin>
            </plugins>
        </build>
    
    </project>
    

    创建log4j2-spring.xml进行配置输出日志信息,如下:

    <?xml version="1.0" encoding="UTF-8"?>
    <Configuration status="OFF" packages="org.graylog2.log4j2">
        <Properties>
            <Property name="LOG_PATTERN">%d{yyyy-MM-dd HH:mm:ss:SSS} - %-5level  - %pid - %t - %c{1.}:%L - %m%n</Property>
        </Properties>
        <Appenders>
            <Console name="Console" target="SYSTEM_OUT" follow="true">
                <ThresholdFilter level="trace" onMatch="ACCEPT" onMismatch="DENY" />
                <PatternLayout pattern="${LOG_PATTERN}"/>
            </Console>
            <GELF name="gelfAppender" server="106.13.35.42" port="12201" hostName="appserver01.example.com">
                <PatternLayout pattern="%logger{36} - %msg%n"/>
                <Filters>
                    <Filter type="MarkerFilter" marker="FLOW" onMatch="DENY" onMismatch="NEUTRAL"/>
                    <Filter type="MarkerFilter" marker="EXCEPTION" onMatch="DENY" onMismatch="ACCEPT"/>
                </Filters>
                <!-- Additional fields -->
                <KeyValuePair key="foo" value="bar"/>
                <KeyValuePair key="jvm" value="${java:vm}"/>
            </GELF>
        </Appenders>
        <Loggers>
            <Root level="info">
                <AppenderRef ref="gelfAppender"/>
                <AppenderRef ref="Console"/>
            </Root>
        </Loggers>
    </Configuration>
    

    这个项目使用的端口号是8888,可以在日志中清晰的看到。

    image

    5. ELK vs Graylog

    这里仅以日志收集为例,简单说一下二者之间的选择,我个人的建议就是取决于现有技术栈,比如现在就有现成的Mongodb,那么选择Graylog可以节省不少成本,ELK类似,不要盲目的追求技术而选择。

    6. 源码

    springboot2_graylog源码地址:https://gitee.com/dalaoyang/springboot_learn/tree/master/springboot2_graylog

    springboot2_graylog_log4j源码地址:https://gitee.com/dalaoyang/springboot_learn/tree/master/springboot2_graylog_log4j

    相关文章

      网友评论

        本文标题:SpringBoot使用Graylog日志收集

        本文链接:https://www.haomeiwen.com/subject/jsrugqtx.html