elk 中使用logstash 模版一些template， ma

先看看遇到哪些坑

类型转换

more than one type

_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x541335cc>], :response=>{"index"=>{"_index"=>"report_log-1", "_type"=>"doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [report_log-1] as the final mapping would have more than 1 type: [doc, logs]"}}}}

上面这两个原因是由多template的index匹配引起的。

logstash 使用es模版的两个纬度

index_patterns匹配

order 优化，默认是0， 值越大，优先级越高

默认使用logstash 模版

查看模版

curl -XGET 127.0.0.1:9200/_template?pretty

写入一个多fieldata段的模版

curl -XPUT '127.0.0.1:9200/_template/logstash' -H 'Content-Type: application/json' -d '{"order":0,"index_patterns":["log-*","nginx*","error_log-*","report_log-*","test_env-log-*","test_env-nginx*","test_env-error_log-*","test_env-report_log-*"],"settings":{"index":{"routing":{"allocation":{"require":{"box_type":"hot"}}},"refresh_interval":"60s","number_of_shards":"1","number_of_replicas":"0"}},"mappings":{"text":{"properties":{"beat.hostname":{"type":"keyword"},"@version":{"type":"keyword"},"beat.name":{"type":"keyword"},"beat.version":{"type":"keyword"},"file":{"type":"keyword"},"function_name":{"type":"keyword"},"host.name":{"type":"text"},"index_date":{"type":"keyword"},"input.type":{"type":"keyword"},"ip":{"type":"keyword"},"level":{"type":"keyword"},"line_no":{"type":"keyword"},"log_body":{"type":"text","norms":false},"message":{"type":"text","norms":false},"module_name":{"type":"keyword"},"process_id":{"type":"keyword"},"prospector.type":{"type":"keyword"},"source":{"type":"keyword"},"thread_id":{"type":"keyword"},"nginx.access.body_bytes_sent":{"type":"integer"},"nginx.access.geoip.city_name":{"type":"keyword"},"nginx.access.geoip.continent_code":{"type":"keyword"},"nginx.access.geoip.country_code2":{"type":"keyword"},"nginx.access.geoip.country_code3":{"type":"keyword"},"nginx.access.geoip.country_name":{"type":"keyword"},"nnginx.access.geoip.ip":{"type":"keyword"},"nnginx.access.geoip.postal_code":{"type":"keyword"},"nginx.access.geoip.region_code":{"type":"keyword"},"nginx.access.geoip.region_name":{"type":"keyword"},"nginx.access.geoip.timezone":{"type":"keyword"},"nginx.access.http_cookie":{"type":"keyword"},"nginx.access.http_host":{"type":"keyword"},"nginx.access.http_refer":{"type":"keyword"},"nginx.access.http_x_forwarded_for":{"type":"keyword"},"nginx.access.http_version":{"type":"keyword"},"nginx.access.method":{"type":"keyword"},"nginx.access.remote_ip":{"type":"keyword"},"nginx.access.request_time":{"type":"float"},"nginx.access.response_code":{"type":"keyword"},"nginx.access.upstream_addr":{"type":"keyword"},"nginx.access.upstream_port":{"type":"keyword"},"nginx.access.upstream_response_time":{"type":"float"},"nginx.access.upstream_status":{"type":"keyword"},"nginx.access.url":{"type":"keyword"},"nginx.access.user_name":{"type":"keyword"},"nginx.access.geoip.location":{"type":"geo_point"},"nginx.access.agent":{"type":"text"},"nginx.access.user_agent.build":{"type":"keyword"},"nginx.access.user_agent.device":{"type":"keyword"},"nginx.access.user_agent.major":{"type":"keyword"},"nginx.access.user_agent.minor":{"type":"keyword"},"nginx.access.user_agent.name":{"type":"keyword"},"nginx.access.user_agent.os":{"type":"keyword"},"nginx.access.user_agent.os_major":{"type":"keyword"},"nginx.access.user_agent.os_minor":{"type":"keyword"},"nginx.access.user_agent.os_name":{"type":"keyword"},"nginx.access.user_agent.patch":{"type":"keyword"},"report_version":{"type":"keyword"},"unix_time":{"type":"long"},"caller_id":{"type":"keyword"},"caller_node":{"type":"keyword"},"callee_id":{"type":"keyword"},"callee_node":{"type":"keyword"},"method":{"type":"keyword"},"ret_code":{"type":"keyword"},"ret_code_percentage":{"type":"integer"},"cost_time":{"type":"float"}}}},"aliases":{}}'