美文网首页
ES解析Nginx日志不同时间格式

ES解析Nginx日志不同时间格式

作者: 野草_疯长 | 来源:发表于2019-10-24 16:04 被阅读0次

nginx有2种时间输出格式,分别是:

$time_iso8601

例:2017-01-17T16:51:42+08:00

$time_local

例:17/Jan/2017:17:14:08 +0800

下面比较 nginx配置中输出日志格式的时间字段在两种格式下的解析方法:

$time_iso8601

log_format json '{"@timestamp":"$time_iso8601",'
                 '"host":"$server_addr",'
                 '"clientip":"$remote_addr",'
                 '"request":"$request",'
                 '"status":"$status",'
                 '"request_method": "$request_method",'
                 '"size":"$body_bytes_sent",'
                 '"request_time":"$request_time",'
                 '"upstreamtime":"$upstream_response_time",'
                 '"upstreamhost":"$upstream_addr",'
                 '"http_host":"$host",'
                 '"url":"$uri",'
                 '"http_forward":"$http_x_forwarded_for",'
                 '"referer":"$http_referer",'
                 '"agent":"$http_user_agent"}';
access_log  /var/log/nginx/access.log json ;

此时,日志中的时间格式为”2017-01-17T16:51:42+08:00”
logstash解析该时间格式配置如下,此时时间戳timestamp采用locals:

filter {
    grok {
        match => { "message" => "%{TIMESTAMP_ISO8601:locals}" }
    }
    date {
        locale => "en"
        match => [ "locals", "ISO8601"]
    }
}

输入:2017-01-17T11:53:13+08:00
输出:

{ 
“@timestamp” => 2017-01-17T00:08:41.000Z, 
“@version” => “1”, 
“host” => “elk.dev”, 
“message” => “2017-01-17T08:08:41+08:00”, 
“locals” => “2017-01-17T08:08:41+08:00”, 
“tags” => [] 
}

$time_local

nginx配置使用该变量时时间格式为“17/Jan/2017:17:14:08 +0800”
此格式相应的logstash配置如下,

filter {
    grok {
        match => ["message", "%{HTTPDATE:logdate}"]
    }
    date {
        locale => "en"
        match => ["logdate", "dd/MMM/yyyy:HH:mm:ss Z"]
    }
}

输入:17/Jan/2017:17:11:10 +0800
输出:

{ 
“@timestamp” => 2017-01-17T09:11:10.000Z, 
“logdate” => “17/Jan/2017:17:11:10 +0800”, 
“@version” => “1”, 
“host” => “elk.dev”, 
“message” => “17/Jan/2017:17:11:10 +0800”, 
“tags” => [] 
}

相关文章

网友评论

      本文标题:ES解析Nginx日志不同时间格式

      本文链接:https://www.haomeiwen.com/subject/ceorvctx.html