ELK常见命令

ELK中常见的命令整理如下：

ES
说明：REST API调用方式为 <REST Verb> /<Index>/<Type>/<ID>

• 查看ES中的index信息： GET /_cat/indices?v

• 查看集群的健康状态： GET /_cat/health?v

• 查看集群的节点信息： GET /_cat/nodes?v

• 创建index：

PUT /customer?pretty
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "index": "customer"
}
PUT /customer/_doc/1?pretty
{
  "name": "xi dada"
}

• 删除index：

DELETE /customer?pretty
DELETE /customer/_doc/1?pretty

• 更新index：

POST /customer/_doc/1/_update?pretty
{
  "doc": {"name":"wang FengFeng", "age":28}
}
POST /customer/_doc/1/_update?pretty
{
  "script": "ctx._source.age += 3"
}

• 查询index信息：
  GET /customer/_doc/2?pretty
  GET /kafka-snail/_search?pretty&q=* 某个index中所有的记录
• 复杂查询index信息+条件：

GET /bank/_search?q=*&sort=account_number:asc&pretty
GET /bank/_search
{
  "query": { "match_all": {} },
  "sort": [
    { "account_number": "asc" }
  ]
}
// 指定查询结果的大小和起始位置
GET /bank/_search
{
  "query": {"match_all": {}},
  "_source": ["account_number","balance"], 
  "sort": [
    { "account_number": "asc" },
    {"balance": "desc"}
  ],
  "from":10,
  "size": 5
}
// 指定多个查询条件，包含与、或、非
GET /bank/_search
{
  "query": { 
    "bool": { 
      "must":[
        {"match":{"address":"mill"}},
        {"match":{"gender":"M"}}
      ],
      "must_not":[
        {"match":{"state":"IL"}}
        ],
      "should": [
        {"match": {"city": "Lopezo"}},
        {"match": {"city": "Urie"}}
      ]
    }
  }
}
//使用where条件，限定字段的范围
GET /bank/_search
{
  "query": {
    "bool": {
      "must": { "match": {"gender":"M"} },
      "filter": {
        "range": {
          "balance": {
            "gte": 20000,
            "lte": 23000
          }
        }
      }
    }
  }
}

• 批量导入或者更新index：
  参见官网 https://www.elastic.co/guide/en/elasticsearch/reference/6.4/docs-bulk.html

// 批量操作
POST /customer/_doc/_bulk?pretty
  {"index":{"_id":"1"}}
  {"name":"John legend"}
  {"index":{"_id":"2"}}
  {"name":"wang jun"}

POST /customer/_doc/_bulk?pretty
{"update":{"_id":"1"}}
{"doc":{"name":"John legend become mengmeng"}}
{"delete":{"_id":"2"}}

• 按indices查看缓存 GET /_stats/request_cache?human
• 按节点查看缓存 GET /_nodes/stats/indices/request_cache?human
• 清理多个index的缓存 POST /index1,inex2/_cache/clear?request=true

Logstash:

• 查看logstash使用的插件信息和版本
  ./bin/logstash-plugin list --verbose
• 卸载logstash使用的某个xx插件
  ./bin/logstash-plugin remove xx
• logstash安装指定版本的插件
  ./bin/logstash-plugin install --version 8.0.6 xx 例如：./bin/logstash-plugin install --version 8.0.6 logstash-input-kafka
• 简单控制台输入输出
  ./bin/logstash -e 'input { stdin {}} output { stdout { } }'
• 配置好Logstash.conf，检查其配置项
  ./bin/logstash -f logstash.conf --config.test_and_exit
• 配置好Logstash.conf，检查其配置项
  ./bin/logstash -f logstash.conf --config.test_and_exit
• 启动logstash，当配置文件修改时，可自动加载
  ./bin/logstash -f logstash.conf --config.reload.automatic
• logstash读取kafka数据存入到ES中的配置kafka-logstash.conf

input {
     kafka {
        bootstrap_servers => "10.194.xxx.yyy:9092,10.xxx.yyy.18:9092,10.xxx.yyy.180:9092"
        auto_offset_reset => "earliest"
        group_id => "logstash23"
    id => "8.0.6"
        client_id => "logstash-5"
        check_crcs => "false"
    topics => ["mda.online"]
        }
}
filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
        source => "clientip"
    }
}
output {
      stdout { }   //会打印出信息
      elasticsearch {
                hosts => "esIp:9200"
                index => "kafka-snail"
        template_overwrite => true
        } 
}

参考文献

• https://www.elastic.co/guide/en/kibana/6.4/production.html kibana真实环境的应用