美文网首页
elk大数据搜索与数据挖掘环境搭建

elk大数据搜索与数据挖掘环境搭建

作者: 兔宝宝在北京 | 来源:发表于2015-12-01 09:56 被阅读0次

1、elasticsearch

(1)下载并安装elasticsearch

  wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.9.zip
  tar zxvf elasticsearch-0.90.9.zip

(2)配置elasticsearch目录下的config/elasticsearch.yml文件,添加以下内容

# Disable HTTP completely:  
#  
http.enabled: (空格)true  
http.cors.enable: (空格)true 

(3)安装elasticsearch head插件

cd elasticsearch安装路径/bin
./plugin -install mobz/elasticsearch-head

(4)安装curl,并配置环境变量

wget http://curl.haxx.se/download/curl-7.20.0.tar.gz
tar -zxf curl-7.20.0.tar.gz
./configure --prefix=/usr/local/curl
make
make install
export PATH=$PATH:/usr/local/curl/bin

(5)进入elasticsearch目录,启动elasticsearch

` cd elasticsearch安装目录  
./bin/elasticsearch  `  

(6)停止elasticsearch

ctrl+c

2、kibana

(1)安装kibana,并将其配置到一个新的目录,以便后续安装niginx服务器运行

wget https://download.elasticsearch.org/kibana/kibana/kibana-3.0.1.tar.gz 
tar zxvf kibana-3.0.1.tar.gz 
mkdir -p /var/www/kibana3 
cp -R ./kibana-3.0.1/* /var/www/kibana3/ 

(2)修改kibana文件目录下的config.js, 保证文件中有如下内容未被注释:

elasticsearch: "http://"+window.location.hostname+":9200"

3、niginx

(1)按照openssl、pcre、zlib的顺序安装niginx的依赖包,并安装niginx

gzip模块需要zlib库
rewrite模块需要pcre库
ssl功能需要openssl库

tar zxvf openssl-fips-2.0.10
cd openssl-fips-2.0.10
./config
make
make install
tar zxvf zlib-1.2.8.tar.gz
cd zlib-1.2.8 
./configure
make
make install
tar zxvf pcre-8.21.tar.gz 
cd pcre-8.21
./configure
make
make install
wget http://nginx.org/download/nginx-1.5.9.tar.gz
tar zxvf nginx-1.5.9.tar.gz
cd  nginx-1.5.9.tar.gz
./configure --with-pcre=../pcre-8.21 --with-zlib=../zlib-1.2.8 --with-     openssl=../openssl-fips-2.0.10
make
make install

(2)检测niginx是否安装成功

cd /usr/local/nginx/sbin
./nginx -t

出现以下消息表示安装成功

nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok 
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful 

(3)配置niginx

配置路径:/usr/local/nginx/conf/nginx.conf
#虚拟主机的配置
server {
listen *:80 ;
server_name localhost;
access_log /usr/local/nginx/logs/access.log;

  location / {  
    root  /var/www/kibana3;  
    index  index.html  index.htm;  
  }  

  location ~ ^/_aliases$ {  
    proxy_pass http://127.0.0.1:9200;  
    proxy_read_timeout 90;  
  }  
  location ~ ^/.*/_aliases$ {  
    proxy_pass http://127.0.0.1:9200;  
    proxy_read_timeout 90;  
  }  
  location ~ ^/_nodes$ {  
    proxy_pass http://127.0.0.1:9200;  
    proxy_read_timeout 90;  
  }  
  location ~ ^/.*/_search$ {  
    proxy_pass http://127.0.0.1:9200;  
    proxy_read_timeout 90;  
  }  
  location ~ ^/.*/_mapping {  
    proxy_pass http://127.0.0.1:9200;  
    proxy_read_timeout 90;  
  }  

  # Password protected end points  
  location ~ ^/kibana-int/dashboard/.*$ {  
    proxy_pass http://127.0.0.1:9200;  
    proxy_read_timeout 90;  
    limit_except GET {  
      proxy_pass http://127.0.0.1:9200;  
      auth_basic "Restricted";  
      auth_basic_user_file /etc/nginx/conf.d    /kibana.myhost.org.htpasswd;  
    }  
  }  
  location ~ ^/kibana-int/temp.*$ {  
    proxy_pass http://127.0.0.1:9200;  
    proxy_read_timeout 90;  
    limit_except GET {  
      proxy_pass http://127.0.0.1:9200;  
      auth_basic "Restricted";  
      auth_basic_user_file /etc/nginx/conf.d    /kibana.myhost.org.htpasswd;  
    }  
  }  
}  

(4)niginx的相关操作

 启动nginx:./nginx
./nginx -s reload:修改配置后重新加载生效
./nginx -s reopen :重新打开日志文件

关闭nginx:
./nginx -s stop:快速停止nginx
                quit :完整有序的停止nginx

查看端口:netstat -ntlp

4、logstash

wget https://download.elasticsearch.org/logstash/logstash/logstash-1.4.2.tar.gz 
tar zxvf logstash-1.4.2.tar.gz  

5、验证安装

(1)进入elasticsearch目录,启动elasticsearch

(2)在logstash目录创建一个文件sample.conf,其内容如下,其中output里将输出定向到两个地方,一个是elasticsearch,一个是本地的console。

input {   
    stdin{}   
}        
output {  
    elasticsearch { host => localhost }  
  
    stdout {  
        codec => rubydebug  
    }  
}  

(3) 在logstash安装目录下执行命令启动logstash

./bin/logstash -f sample.conf  

(4)启动后,在屏幕输入hello,可看到屏幕有message输出,并且打开浏览器,输入http://localhost,则可以看到kibana的页面。

相关文章

网友评论

      本文标题:elk大数据搜索与数据挖掘环境搭建

      本文链接:https://www.haomeiwen.com/subject/eijjhttx.html