Linux安装Logstash

前言

logstash是ELK日志系统中的一部分，主要承担将收集完成日志进行过滤，并且输出到es的职责。

Logstash的事件（logstash将数据流中等每一条数据称之为一个event）处理流水线有三个主要角色完成：inputs –> filters –> outputs：

inpust：必须，负责产生事件（Inputs generate events），常用：File、syslog、redis、beats（如：Filebeats）

filters：可选，负责数据处理与转换（filters modify them），常用：grok、mutate、drop、clone、geoip

outpus：必须，负责数据输出（outputs ship them elsewhere），常用：elasticsearch、file、graphite、statsd。

需求：需要把数据库oracle 里的数据同步到es中。所以考虑要用到logstasg里的插件logstash-input-jdbc 所以先安装logstash

1. 下载 （linux版本）：（版本 7.3.2）

官网地址：https://www.elastic.co/downloads/logstash

本人在网盘中下载比较快，国外那个太慢了，一直超时。 地址 https://pan.baidu.com/s/1CYsQvfhLoRAGUA6-hRRuqw 提取码：ats6

2:解压安装

命令： tar -zxvf logstash-7.3.2.tar.gz

启动 进去到bin目录下 执行 

./logstash -e 'input { stdin { } } output { stdout {} }'

控制台输出：

Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

WARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/home/logstash-7.3.2/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.FileDescriptor.fd

WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules

WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations

WARNING: All illegal access operations will be denied in a future release

Thread.exclusive is deprecated, use Thread::Mutex

Sending Logstash logs to /home/logstash-7.3.2/logs which is now configured via log4j2.properties

[2020-03-01T07:10:08,891][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/home/logstash-7.3.2/data/queue"}

[2020-03-01T07:10:08,922][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/home/logstash-7.3.2/data/dead_letter_queue"}

[2020-03-01T07:10:09,491][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified

[2020-03-01T07:10:09,513][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.3.2"}

[2020-03-01T07:10:09,562][INFO ][logstash.agent          ] No persistent UUID file found. Generating new UUID {:uuid=>"cdb36cc9-faca-451d-8d3d-b9d447141467", :path=>"/home/logstash-7.3.2/data/uuid"}

[2020-03-01T07:10:11,071][INFO ][org.reflections.Reflections] Reflections took 79 ms to scan 1 urls, producing 19 keys and 39 values

[2020-03-01T07:10:12,198][WARN ][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge] A gauge metric of an unknown type (org.jruby.RubyArray) has been create for key: cluster_uuids. This may result in invalid serialization.  It is recommended to log an issue to the responsible developer/development team.

[2020-03-01T07:10:12,206][INFO ][logstash.javapipeline    ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, :thread=>"#<Thread:0x7e41ce73 run>"}

[2020-03-01T07:10:28,362][INFO ][logstash.javapipeline    ] Pipeline started {"pipeline.id"=>"main"}

The stdin plugin is now waiting for input:

[2020-03-01T07:10:28,476][INFO ][logstash.agent          ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}

[2020-03-01T07:10:28,811][INFO ][logstash.agent          ] Successfully started Logstash API endpoint {:port=>9600}

---

表示启动成功.

测试 输入数据

helloword

{

      "@version" => "1",

          "host" => "0.0.0.0",

      "message" => "helloword",

    "@timestamp" => 2020-02-29T23:11:07.342Z

}