ES的安装和简单的配置

在https://www.elastic.co/cn/products/elastic-stack
下载特定的ES版本。

ES目录.png image.png

bin/elasticsearch-plugin lists 查看安装的插件
bin/elasticsearch-plugin install analysis-icu ## 安装一个插件测试

插件安装简单，通过安装插件实现数据同步，安全策略。

2.1 本机多实例的方式启动

bin/elasticsearch -E node.name=node1 -E cluster.name=guodong -E path.data=node1_data -d
bin/elasticsearch -E node.name=node2 -E cluster.name=guodong -E path.data=node2_data -d

查看运行的节点.png

Kibana 初体验

下载Kibana ，dashboard。

dev tool.png

http://localhost:5601/app/kibana#/dev_tools/console?_g=()

devtool为调试工具

logstash工具导入测试数据

使用了一个movieslens的测试数据集。

./bin/logstash -f movielens/logstash.conf

config文件内容如下。

input {
  file {
    path => "/Users/yiruan/dev/elk7/logstash-7.0.1/bin/movies.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
    separator => ","
    columns => ["id","content","genre"]
  }

  mutate {
    split => { "genre" => "|" }
    remove_field => ["path", "host","@timestamp","message"]
  }

  mutate {

    split => ["content", "("]
    add_field => { "title" => "%{[content][0]}"}
    add_field => { "year" => "%{[content][1]}"}
  }

  mutate {
    convert => {
      "year" => "integer"
    }
    strip => ["title"]
    remove_field => ["path", "host","@timestamp","message","content"]
  }

}
output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "movies"
     document_id => "%{id}"
   }
  stdout {}
}

对应的CSV文件部分内容如下

movieId,title,genres
1,Toy Story (1995),Adventure|Animation|Children|Comedy|Fantasy
2,Jumanji (1995),Adventure|Children|Fantasy
3,Grumpier Old Men (1995),Comedy|Romance
4,Waiting to Exhale (1995),Comedy|Drama|Romance
5,Father of the Bride Part II (1995),Comedy
6,Heat (1995),Action|Crime|Thriller