Elk

elk : elasticsearch logstash kibana
elasticsearch: 部署
1.下载并安装GPG Key
[root@elk-node1 ~]# rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

2.添加yum仓库
[root@elk-node1 ~]# vim /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-2.x]
name=Elasticsearch repository for 2.x packages
baseurl=http://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=0
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

yum install -y elasticsearch

yum install -y java java -version

mkdir -p /data/es-data

vim /etc/elasticsearch/elasticsearch.yml
cluster.name: wingcluster                  # 组名（同一个组，组名必须一致）
node.name: elk-node1                        # 节点名称，建议和主机名一致
path.data: /data/es-data                     # 数据存放的路径
path.logs: /var/log/elasticsearch/        # 日志存放的路径               elasticsearch自己的日志
bootstrap.mlockall: true                      # 锁住内存，不被使用到交换分区去
network.host: 0.0.0.0                          # 网络设置       监听本地所有网卡 0.0.0.0
http.port: 9200                                    # 端口

从服务器要在配置文件中追加入如下内容

discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["172.16.113.155", "172.16.113.156"]    两台各自的IP

chown -R elasticsearch.elasticsearch /data/

systemctl start elasticsearch

1.安装head插件

[root@elk-node1 src]# /usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head
[root@elk-node1 head]# chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/plugins
[root@elk-node1 _site]# systemctl restart elasticsearch

2.安装kopf监控插件

[root@elk-node1 src]# /usr/share/elasticsearch/bin/plugin install lmenezes/elasticsearch-kopf

[root@elk-node1 kopf]# chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/plugins

[root@elk-node1 _site]# systemctl restart elasticsearch

http://172.16.113.155:9200/_plugin/kopf/#!/cluster

logstash

1.下载并安装GPG Key
[root@elk-node1 ~]# rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

2.添加yum仓库
[root@hadoop-node1 ~]# vim /etc/yum.repos.d/logstash.repo
[logstash-2.1]
name=Logstash repository for 2.1.x packages
baseurl=http://packages.elastic.co/logstash/2.1/centos
gpgcheck=0
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

3.安装logstash
[root@elk-node1 ~]# yum install -y logstash

使用rubydebug和写到elasticsearch中的区别：其实就在于后面标准输出的区别，前者使用codec；后者使用elasticsearch

logstash 日志收集

1）logstash的配置
简单的配置方式：

[root@elk-node1 ~]# vim /etc/logstash/conf.d/01-logstash.conf
input { stdin { } }
output {
        elasticsearch { hosts => ["192.168.1.160:9200"]}
        stdout { codec => rubydebug }
}
它的执行：
[root@elk-node1 ~]# /opt/logstash/bin/logstash -f /etc/logstash/conf.d/01-logstash.conf
Settings: Default filter workers: 1
Logstash startup completed
beijing                                                #输入内容
{                                                       #输出下面信息
             "message" => "beijing",
            "@version" => "1",
       "@timestamp" => "2016-11-11T07:41:48.401Z",
                   "host" => "elk-node1"
}

2）收集系统日志

[root@elk-node1 ~]# vim  file.conf
input {
    file {
      path => "/var/log/messages"
      type => "system"
      start_position => "beginning"
    }
}
output {
    elasticsearch {       masterIP
           hosts => ["172.16.113.155:9200"]
           index => "system-%{+YYYY.MM.dd}"    
    }
}

执行上面日志信息的收集，如下，这个命令会一直在执行中，表示日志在监控收集中；如果中断，就表示日志不在收集！所以需要放在后台执行~

[root@elk-node1 ~]# /opt/logstash/bin/logstash -f file.conf &

Kibana安装配置

下载kibana安装包wget https://download.elastic.co/kibana/kibana/kibana-4.3.1-linux-x64.tar.gz   
解压到/usr/local/ 
                tar zxf kibana-4.3.1-linux-x64.tar.gz -C /usr/local       做一个软连接 
                ln -s /usr/local/kibana-4.3/1-linux-x64/ /usr/local/kibana


2）修改配置文件：
[root@elk-node1 config]# pwd
/usr/local/kibana/config
[root@elk-node1 config]# cp kibana.yml kibana.yml.bak
[root@elk-node1 config]# vim kibana.yml 
server.port: 5601
server.host: "0.0.0.0"   后端服务器的主机默认localhost                          

elasticsearch.url: "http://192.168.1.160:9200"默认http://localhost:9200用于查看elasticsearch 
的实例URL

kibana.index: ".kibana"

因为他一直运行在前台，要么选择开一个窗口，要么选择使用screen。安装并使用screen启动kibana： 或



（3）访问kibana：http://172.16.113.155:5601/