【elasticsearch】3、logstash安装

logstash下载

logstash下载

注意：logstash和es以及kibana的版本需要是一致的

准备导入数据

movielens-电影数据

image.png

配置logstash.conf下面的input为你下载的电影数据

input {
  file {
    path => "/Users/cutie/workspace/elasticsearch/logstash-7.6.0/movie-data/ml-latest-small/movies.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
    separator => ","
    columns => ["id","content","genre"]
  }

  mutate {
    split => { "genre" => "|" }
    remove_field => ["path", "host","@timestamp","message"]
  }

  mutate {
    split => ["content", "("]
    add_field => { "title" => "%{[content][0]}"}
    add_field => { "year" => "%{[content][1]}"}
  }

  mutate {
    convert => {
      "year" => "integer"
    }
    strip => ["title"]
    remove_field => ["path", "host","@timestamp","message","content"]
  }

}
output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "movies"
     document_id => "%{id}"
   }
  stdout {}
}

复制logstash.sample.conf为logstash.conf

sudo bin/logstash -f config/logstash.conf #启动logstash

注意：在执行logstash时，需要使用sudo，否则命令执行完成后没有权限操作相关文件