1. 安装JDK（tar.gz版本）
官网下载： http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
解压到指定的目录（filename替换为你的tar包）：
tar -zxvf filename -C /usr/local/java/
修改配置文件：
vim /etc/profile
添加：
export JAVA_HOME=/usr/java/jdk1.8.0_191-amd64
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
使生效：
source /etc/profile
校验：
java -version
若成功，则显示 java 版本：
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
2. filebeat安装
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.5.2-linux-x86_64.tar.gz
tar xzvf filebeat-6.5.2-linux-x86_64.tar.gz
修改 filebeat.yml （注释除kafka以外的输出源）
#=========================== Filebeat inputs =============================
filebeat.inputs:
# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.
- type: log
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /*/log/*
fields:
testname: xxxnn
fields_under_root: true
#----------------------------- kafka output --------------------------------
output.kafka:
enabled: true
hosts: ["localhost:9092"]
max_retries: 5
timeout: 300
topic: "filebeat"
配置验证:
./filebeat test config -c filebeat.yml
3. kafka、zookeeper安装
docker 镜像拉取
docker pull wurstmeister/zookeeper:latest
docker pull wurstmeister/kafka:latest
连接、启动服务
sudo docker run -d --name zookeeper -p 2181 -t wurstmeister/zookeeper:latest
sudo docker run -d --name kafka --publish 9092:9092 --link zookeeper --env KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 --env KAFKA_ADVERTISED_HOST_NAME=127.0.0.1 --env KAFKA_ADVERTISED_PORT=9092 wurstmeister/kafka:latest
进入容器查看配置
docker exec -it kafka bash
4. 安装logstash
下载公共签名文件：
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
在 /etc/yum.repos.d/ 路径下添加你的 .repo 文件，比如 logstash.repo
[logstash-6.x]
name=Elastic repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
添加完依赖以后，你可以使用yum进行安装了
sudo yum install logstash
添加配置文件test.conf
input {
kafka {
bootstrap_servers => "127.0.0.1:9092"
topics => ["filebeat"]
group_id => "test-consumer-group"
codec => "plain"
consumer_threads => 1
decorate_events => true
}
}
output {
elasticsearch {
hosts => ["127.0.0.1:9200"]
index => "test" workers => 1
}
}
测试配置文件
./logstash -f test.conf --config.test_and_exit
5. elasticsearch 安装
拉取镜像
docker pull docker.elastic.co/elasticsearch/elasticsearch-oss:6.3.2
6. 服务启动
1）elasticsearch
sudo docker run -d --name es -p 9200:9200 -t docker.elastic.co/elasticsearch/elasticsearch-oss:6.3.2
2）logstash
./logstash -f test.conf --config.reload.automatic
3) kafka
上文已经处于启动状态
4）filebeat
./filebeat -e -c filebeat.yml -d "publish"
7. 添加 kibana 可视化操作界面
下载kibana
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.3.2-linux-x86_64.tar.gz
tar -zxvf kibana-6.3.2-linux-x86_64.tar.gz
修改配置文件
vim config/kibana.yml
# 放开注释,将默认配置改成如下：
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://192.168.202.128:9200"
kibana.index: ".kibana"
启动
./kibana
至此单机版的安装已经全部完成
网友评论