7.12.0 es+kibana+logstash

作者: 码农工号9527 | 来源:发表于2021-06-23 21:15 被阅读0次

7.12.0 es+kibana+logstash
logstash导入文件日志到es
docekr 安装elasticsearch:7.12.0
Elasticsearch 常用查询命令
gitlab版本升级并docker化
CentOS7 Docker部署Kibana+ElasticSe

1. 安装java环境【说是Elasticsearch 7.0 开始，内置了 Java 环境，反正也不麻烦，在装下吧】

1.1 下载相应版本的压缩包

下载压缩包可以通过官网下载
下载前需先输入“uname -r”查看自身Linux的版本。

[root@localhost local]# uname -r
3.10.0-957.12.2.el7.x86_64

下载jdk

[root@localhost download]# wget https://download.oracle.com/otn-pub/java/jdk/16.0.1+9/7147401fd7354114ac51ef3e1328291f/jdk-16.0.1_linux-x64_bin.tar.gz

java官网应该现在已经做了处理，下载需要携带参数，完整参数格式如下： https://download.oracle.com/otn-pub/java/jdk/16.0.2+7/d4a915d82b4c4fbb9bde534da945d746/jdk-16.0.2_linux-x64_bin.tar.gz?AuthParam=1630152782_c972da8e1ac38fe0fe40fcdc1fe9575c
因此建议直接去官网找到对应需要的版本下载地址在复制下来到 linux 去 wget

1.2 解压缩并移至相应目录

[root@localhost download]# tar -zxvf jdk-16.0.1_linux-x64_bin.tar.gz
...
[root@localhost download]# mv jdk-16.0.1 java
[root@localhost download]# mv java /usr/local/
[root@localhost local]# ll /usr/local/
total 0
drwxr-xr-x.  2 root root   6 Apr 11  2018 bin
drwxr-xr-x.  2 root root   6 Apr 11  2018 etc
drwxr-xr-x.  2 root root   6 Apr 11  2018 games
drwxr-xr-x. 10 root root 272 May  6 15:05 go
drwxr-xr-x.  2 root root   6 Apr 11  2018 include
drwxr-xr-x.  9 root root 107 Jun 17 03:00 java
drwxr-xr-x.  2 root root   6 Apr 11  2018 lib
drwxr-xr-x.  2 root root   6 Apr 11  2018 lib64
drwxr-xr-x.  2 root root   6 Apr 11  2018 libexec
drwxr-xr-x.  4 root root  50 Jun  7 06:49 protoc
drwxr-xr-x.  2 root root   6 Apr 11  2018 sbin
drwxr-xr-x.  5 root root  49 Jun  1  2019 share
drwxr-xr-x.  2 root root   6 Apr 11  2018 src

1.3 配置环境变量
通过vim /etc/profile命令，打开etc目录下的环境变量配置文件profile，最后面追加如下内容

export JAVA_HOME=/usr/local/java
export PATH=$JAVA_HOME/bin:$PATH

通过source /etc/profile命令更新配置文件，并验证java版本

[root@localhost local]# source /etc/profile
[root@localhost local]# java -version
java version "16.0.1" 2021-04-20
Java(TM) SE Runtime Environment (build 16.0.1+9-24)
Java HotSpot(TM) 64-Bit Server VM (build 16.0.1+9-24, mixed mode, sharing)

2. 安装elasticsearch

2.1 下载

[root@localhost local]# cd /data/download/
[root@localhost download]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.12.0-linux-x86_64.tar.gz
--2021-06-17 06:34:04--  https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.2-linux-x86_64.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 34.120.127.130, 2600:1901:0:1d7::
Connecting to artifacts.elastic.co (artifacts.elastic.co)|34.120.127.130|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 318808405 (304M) [application/x-gzip]
Saving to: ‘elasticsearch-7.10.2-linux-x86_64.tar.gz’

100%[=====================================================================================================================================================================================================================================>] 318,808,405 27.9MB/s   in 20s

2021-06-17 06:34:25 (15.1 MB/s) - ‘elasticsearch-7.10.2-linux-x86_64.tar.gz’ saved [318808405/318808405]

[root@localhost download]# ll
total 613084
-rw-r--r--. 1 root root 318808405 Jan 14 13:18 elasticsearch-7.12.0-linux-x86_64.tar.gz
-rw-r--r--. 1 root root 129044044 May  6 18:04 go1.16.4.linux-amd64.tar.gz
-rw-r--r--. 1 root root 178276087 Mar 31 17:39 jdk-16.0.1_linux-x64_bin.tar.gz
-rw-r--r--. 1 root root   1660909 Jun  2 21:06 protoc-3.17.2-linux-x86_64.zip

2.2 解压并移动到指定目录

[root@localhost download]# tar -zxvf elasticsearch-7.12.0-linux-x86_64.tar.gz
[root@localhost download]# mv elasticsearch-7.12.0 /usr/local/elasticsearch
[root@localhost download]# cd /usr/local/elasticsearch/
[root@localhost elasticsearch]# ll
total 560
drwxr-xr-x.  2 root root   4096 Mar 18 14:21 bin
drwxr-xr-x.  3 root root    169 Jun 17 20:57 config
drwxr-xr-x.  9 root root    107 Mar 18 14:21 jdk
drwxr-xr-x.  3 root root   4096 Mar 18 14:21 lib
-rw-r--r--.  1 root root   3860 Mar 18 14:15 LICENSE.txt
drwxr-xr-x.  2 root root      6 Mar 18 14:19 logs
drwxr-xr-x. 60 root root   4096 Mar 18 14:22 modules
-rw-r--r--.  1 root root 545323 Mar 18 14:19 NOTICE.txt
drwxr-xr-x.  2 root root      6 Mar 18 14:19 plugins
-rw-r--r--.  1 root root   7263 Mar 18 14:14 README.asciidoc

Elasticsearch 目录、配置文件说明、注意点，这里大概解答下各个目录、配置文件的作用:

目录	配置文件	描述
bin		放置脚本文件，如启动脚本 elasticsearch, 插件安装脚本等。
config	elasticserch.yml	elasticsearch 配置文件，如集群配置、jvm 配置等。
jdk		java 运行环境
data	path.data	数据持久化文件
lib		依赖的相关类库
logs	path.log	日志文件
modules		包含的所有 ES 模块
plugins		包含的所有已安装的插件

注意点:

有些童鞋的机器内存可能不够，就需要修改 JVM 参数，配置文件路径为 config/jvm.options，ES V7.1 版本默认为 1g, 老版本为2g, 你可以自行修改。
Xmx 和Xms 数值请设置相同；
Xmx 不要超过机器内存的 50%；
内存总量不要超过 30GB, 参见官方文档 https://www.elastic.co/cn/blog/a-heap-of-trouble；

2.3 创建用户并修改文件所属用户用户组

es用户密码设置为 xxxxxx

[root@localhost elasticsearch]# cd ..
[root@localhost local]# useradd es
[root@localhost local]# passwd es
Changing password for user es.
New password:
BAD PASSWORD: The password contains the user name in some form
Retype new password:
passwd: all authentication tokens updated successfully.
[root@localhost local]# chown -R es:es elasticsearch/
[root@localhost local]# ll
total 0
total 4
drwxr-xr-x.  2 root root    89 Jun 17 16:13 bin
drwxr-xr-x.  9 es   es     155 Mar 18 14:21 elasticsearch
drwxr-xr-x.  2 root root     6 Apr 11  2018 etc
drwxr-xr-x.  6 root root    56 Jun 17 15:54 freetype
drwxr-xr-x.  2 root root     6 Apr 11  2018 games
drwxr-xr-x. 10 root root   272 May  6 23:05 go
drwxr-xr-x.  3 root root   122 Jun 17 15:53 include
drwxr-xr-x.  9 root root   107 Jun 17 11:00 java
drwxr-xr-x.  3 root root  4096 Jun 17 15:53 lib
drwxr-xr-x.  2 root root     6 Apr 11  2018 lib64
drwxr-xr-x.  2 root root     6 Apr 11  2018 libexec
drwxr-xr-x.  3 root root    18 Jun 17 15:53 man
drwxr-xr-x. 14 root mysql  216 Jun 17 16:05 mysql
drwxr-xr-x. 11 root root   151 Jun 17 16:15 nginx
drwxr-xr-x. 10 root root   102 Jun 17 16:13 php
drwxr-xr-x.  4 root root    50 Jun  7 14:49 protoc
drwxr-xr-x.  2 root root     6 Apr 11  2018 sbin
drwxr-xr-x.  8 root root    89 Jun 17 15:53 share
drwxr-xr-x.  2 root root     6 Apr 11  2018 src

2.4 启动es

配置外网访问

[es@localhost elasticsearch]$ vim config/elasticsearch.yml

修改配置如下：

......
node.name: node-1
......
network.host: 192.168.33.13
......
http.port: 9200
......
cluster.initial_master_nodes: ["node-1"]
......

测试启动

[root@localhost elasticsearch]# su es
[es@localhost elasticsearch]$ bin/elasticsearch

网页页面访问 http://localhost:9200/?pretty 返回如下则启动成功：

{
  "name" : "node-1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "9DHKf7XhTXOKZh1EQ5tbUw",
  "version" : {
    "number" : "7.12.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "78722783c38caa25a70982b5b042074cde5d3b3a",
    "build_date" : "2021-03-18T06:17:15.410153305Z",
    "build_snapshot" : false,
    "lucene_version" : "8.8.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"

3. 安装kibana

3.1下载

[root@localhost download]# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.12.0-linux-x86_64.tar.gz
--2021-06-17 21:21:10--  https://artifacts.elastic.co/downloads/kibana/kibana-7.12.0-linux-x86_64.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 34.120.127.130, 2600:1901:0:1d7::
Connecting to artifacts.elastic.co (artifacts.elastic.co)|34.120.127.130|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 281022115 (268M) [application/x-gzip]
Saving to: ‘kibana-7.12.0-linux-x86_64.tar.gz’

100%[=====================================================================================================================================================================================================================================>] 281,022,115 16.5MB/s   in 23s

2021-06-17 21:21:34 (11.6 MB/s) - ‘kibana-7.12.0-linux-x86_64.tar.gz’ saved [281022115/281022115]

3.2 解压并移动到指定目录

[root@localhost download]# tar -zxvf kibana-7.12.0-linux-x86_64.tar.gz
[root@localhost download]# mv kibana-7.12.0-linux-x86_64 /usr/local/kibana
[root@localhost download]# cd /usr/local/kibana/
[root@localhost kibana]# ll
total 1448
drwxr-xr-x   2 root root      94 Mar 18 13:56 bin
drwxr-xr-x   2 root root      44 Mar 18 13:55 config
drwxr-xr-x   2 root root       6 Mar 18 13:55 data
-rw-r--r--   1 root root    3860 Mar 18 13:55 LICENSE.txt
drwxr-xr-x   6 root root     108 Mar 18 13:55 node
drwxr-xr-x 831 root root   24576 Mar 18 13:55 node_modules
-rw-r--r--   1 root root 1428396 Mar 18 13:55 NOTICE.txt
-rw-r--r--   1 root root     740 Mar 18 13:55 package.json
drwxr-xr-x   2 root root       6 Mar 18 13:55 plugins
-rw-r--r--   1 root root    3968 Mar 18 13:55 README.txt
drwxr-xr-x  12 root root     187 Mar 18 13:55 src
drwxr-xr-x   3 root root      79 Mar 18 13:55 x-pack

3.3 配置Kibana，更多配置： kibana配置文件

vim config/kibana.yml ，配置elasticsearch地址和kibana地址信息

......
server.host: "192.168.33.13"
......
elasticsearch.hosts: ["http://192.168.33.13:9200"]
......
i18n.locale: "zh-CN"

3.4 启动

不能使用root启动

[root@localhost kibana]# bin/kibana
Kibana should not be run as root.  Use --allow-root to continue.

修改kibana的用户权限为es，接着在切到es用户启动

[root@localhost kibana]# ll
total 1448
drwxr-xr-x   2 root root      94 Mar 18 13:56 bin
drwxr-xr-x   2 root root      44 Jun 17 21:27 config
drwxr-xr-x   2 root root       6 Mar 18 13:55 data
-rw-r--r--   1 root root    3860 Mar 18 13:55 LICENSE.txt
drwxr-xr-x   6 root root     108 Mar 18 13:55 node
drwxr-xr-x 831 root root   24576 Mar 18 13:55 node_modules
-rw-r--r--   1 root root 1428396 Mar 18 13:55 NOTICE.txt
-rw-r--r--   1 root root     740 Mar 18 13:55 package.json
drwxr-xr-x   2 root root       6 Mar 18 13:55 plugins
-rw-r--r--   1 root root    3968 Mar 18 13:55 README.txt
drwxr-xr-x  12 root root     187 Mar 18 13:55 src
drwxr-xr-x   3 root root      79 Mar 18 13:55 x-pack
[root@localhost kibana]# chown -R es:es /usr/local/kibana/
[root@localhost kibana]# ll
total 1448
drwxr-xr-x   2 es es      94 Mar 18 13:56 bin
drwxr-xr-x   2 es es      44 Jun 17 21:27 config
drwxr-xr-x   2 es es       6 Mar 18 13:55 data
-rw-r--r--   1 es es    3860 Mar 18 13:55 LICENSE.txt
drwxr-xr-x   6 es es     108 Mar 18 13:55 node
drwxr-xr-x 831 es es   24576 Mar 18 13:55 node_modules
-rw-r--r--   1 es es 1428396 Mar 18 13:55 NOTICE.txt
-rw-r--r--   1 es es     740 Mar 18 13:55 package.json
drwxr-xr-x   2 es es       6 Mar 18 13:55 plugins
-rw-r--r--   1 es es    3968 Mar 18 13:55 README.txt
drwxr-xr-x  12 es es     187 Mar 18 13:55 src
drwxr-xr-x   3 es es      79 Mar 18 13:55 x-pack

4. 安装logstash

4.1 下载

[root@localhost download]# rm -f logstash-7.12.0-linux-x86_64.tar.gz
[root@localhost download]# wget https://artifacts.elastic.co/downloads/logstash/logstash-7.12.0-linux-x86_64.tar.gz
--2021-06-17 21:49:25--  https://artifacts.elastic.co/downloads/logstash/logstash-7.12.0-linux-x86_64.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 34.120.127.130, 2600:1901:0:1d7::
Connecting to artifacts.elastic.co (artifacts.elastic.co)|34.120.127.130|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 368429061 (351M) [application/x-gzip]
Saving to: ‘logstash-7.12.0-linux-x86_64.tar.gz’

100%[=====================================================================================================================================================================================================================================>] 368,429,061 18.9MB/s   in 45s

2021-06-17 21:50:11 (7.87 MB/s) - ‘logstash-7.12.0-linux-x86_64.tar.gz’ saved [368429061/368429061]

4.2 解压并移动到指定目录

[root@localhost local]# tar -zxvf logstash-7.12.0-linux-x86_64.tar.gz
[root@localhost download]# mv logstash-7.12.0 /usr/local/logstash
[root@localhost download]# cd /usr/local/
[root@localhost local]# ll
total 4
drwxr-xr-x.  2 root root    89 Jun 17 16:13 bin
drwxr-xr-x. 10 es   es     167 Jun 17 21:00 elasticsearch
drwxr-xr-x.  2 root root     6 Apr 11  2018 etc
drwxr-xr-x.  6 root root    56 Jun 17 15:54 freetype
drwxr-xr-x.  2 root root     6 Apr 11  2018 games
drwxr-xr-x. 10 root root   272 May  6 23:05 go
drwxr-xr-x.  3 root root   122 Jun 17 15:53 include
drwxr-xr-x.  9 root root   107 Jun 17 11:00 java
drwxr-xr-x  10 es   es     210 Jun 17 21:22 kibana
drwxr-xr-x.  3 root root  4096 Jun 17 15:53 lib
drwxr-xr-x.  2 root root     6 Apr 11  2018 lib64
drwxr-xr-x.  2 root root     6 Apr 11  2018 libexec
drwxr-xr-x  13 root root   266 Jun 17 21:50 logstash
drwxr-xr-x.  3 root root    18 Jun 17 15:53 man
drwxr-xr-x. 14 root mysql  216 Jun 17 16:05 mysql
drwxr-xr-x. 11 root root   151 Jun 17 16:15 nginx
drwxr-xr-x. 10 root root   102 Jun 17 16:13 php
drwxr-xr-x.  4 root root    50 Jun  7 14:49 protoc
drwxr-xr-x.  2 root root     6 Apr 11  2018 sbin
drwxr-xr-x.  8 root root    89 Jun 17 15:53 share
drwxr-xr-x.  2 root root     6 Apr 11  2018 src

使用logstash定时读取log文件，并插入mysql数据库中，output使用logstash-output-jdbc插件。该插件不是默认安装的，需要使用命令：bin/logstash-plugin install logstash-output-jdbc去官方拉取。

[root@localhost logstash]# ./bin/logstash-plugin install logstash-output-jdbc
Using bundled JDK: /usr/local/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Validating logstash-output-jdbc
Installing logstash-output-jdbc
Installation successful

需要准备一个mysql-connector-java-x.x.x.jar对应版本的jar，这里使用的是mysql-connector-java-5.1.6.jar。

这个可以去官网下载：https://dev.mysql.com/downloads/connector/j/5.1.html

我本地的mysql是5.7版本，下载的是mysql-connector-java-5.1.49

[root@localhost logstash]# cd /data/download/
[root@localhost download]# wget https://downloads.mysql.com/archives/get/p/3/file/mysql-connector-java-5.1.49.tar.gz
[root@localhost download]# tar -zxvf mysql-connector-java-5.1.49.tar.gz
[root@localhost download]# cd mysql-connector-java-5.1.49/
[root@localhost download]# cp mysql-connector-java-5.1.49.jar /usr/local/logstash/logstash-core/lib/jars/
[root@localhost download]# cd /usr/local/logstash/

4.3 配置并启动

# 创建自定义的配置文件目录
[root@localhost logstash]# mkdir sync-diy
# 创建用于记录增量复制位置的文件
[root@localhost logstash]# touch sync-diy/account_track_record
# 创建logstash启动的配置文件
[root@localhost logstash]# touch sync-diy/logstash-mysql-es.conf

sync-diy/logstash-mysql-es.conf文件的内容如下

input {
  stdin{}
  jdbc {
    # mysql相关jdbc配置
    jdbc_connection_string => "jdbc:mysql://localhost:3306/db_name?characterEncoding=utf8&serverTimezone=Asia/Shanghai"
    jdbc_user => "db_user"
    jdbc_password => "db_password"

    # jdbc连接mysql驱动的文件目录
    jdbc_driver_library => "/usr/local/logstash/logstash-core/lib/jars/mysql-connector-java-5.1.49.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    #数据库重连尝试次数
    connection_retry_attempts=>"3"
    #判断数据库连接是否可用，默认false不开启
    jdbc_validate_connection=>"true"
    #数据库连接可用校验超时时间，默认3600S
    jdbc_validation_timeout=>"3600"
    #开启分页查询（默认false不开启）
    jdbc_paging_enabled => true
    jdbc_page_size => "5000"
    jdbc_default_timezone =>"Asia/Shanghai"

    # mysqlSQL语句
    statement => "SELECT user_id, reg_time, 
CASE WHEN JSON_VALID(ext) THEN IFNULL(JSON_UNQUOTE(json_extract(ext, '$.imei')), '') ELSE '' END AS imei,
CASE WHEN JSON_VALID(ext) THEN IFNULL(JSON_UNQUOTE(json_extract(ext, '$.oaid')), '') ELSE '' END AS oaid,
CASE WHEN JSON_VALID(ext) THEN IFNULL(JSON_UNQUOTE(json_extract(ext, '$.idfa')), '') ELSE '' END AS idfa,
CASE WHEN JSON_VALID(ext) THEN IFNULL(JSON_UNQUOTE(json_extract(ext, '$.idfv')), '') ELSE '' END AS idfv
FROM account WHERE reg_time >= :sql_last_value order by reg_time asc"

    # 也可以使用文件
    # statement_filepath => "./config/jdbc.sql"

    #Valuecanbeanyof:fatal,error,warn,info,debug，默认info；
    sql_log_level=>warn

    # 这里类似crontab,可以定制定时操作，比如每分钟执行一次同步(分 时 天 月 年)
    schedule => "* * * * *"
    #type => "jdbc"

    # 是否记录上次执行结果, 如果为真,将会把上次执行到的 tracking_column 字段的值记录下来,保存到 last_run_metadata_path 指定的文件中
    record_last_run => true

    # 是否需要记录某个column 的值,如果record_last_run为真,可以自定义我们需要 track 的 column 名称，此时该参数就要为 true. 否则默认 track 的是 timestamp 的值.
    use_column_value => true

    # 如果 use_column_value 为真,需配置此参数. track 的数据库 column 名,该 column 必须是递增的. 一般是mysql主键
    tracking_column => "reg_time"

    #Valuecanbeanyof:numeric,timestamp，Defaultvalueis"numeric"
    tracking_column_type => "timestamp"

    #record_last_run上次数据存放位置
    last_run_metadata_path => "/usr/local/logstash/sync-diy/account_track_record"

    # 是否清除 last_run_metadata_path 的记录,如果为真那么每次都相当于从头开始查询所有的数据库记录
    clean_run => false

    #是否将 字段(column) 名称转小写
    lowercase_column_names => false
  }
}

output {
  elasticsearch {
    hosts => "192.168.33.13:9200"
    index => "sdk_account_device_info"
    # 对应表的主键
    document_id => "%{user_id}"
    template_overwrite => true
  }
  
  # 这里输出调试，正式运行时可以注释掉
  stdout {
      codec => json_lines
  }
}

启动

[root@localhost logstash]# ./bin/logstash -f ./sync-diy/logstash-mysql-es.conf

多表同步

规则如下:
一个表,一个配置
多个表,多个配置
需要同步多少表,就需要加多少配置

当然配置的内容都差不多,改的地方是查询的表名,和es的索引以及类型的名称
进入logstash/config目录,修改配置文件vim pipelines.yml编辑文件
直接到最后,添加配置

- pipeline.id: table1
  path.config: "/usr/local/software/my/logstash-6.7.2/config/user.conf"
- pipeline.id: table2
  path.config: "/usr/local/software/my/logstash-6.7.2/config/goods.conf"

启动方式稍有改变,进入bin目录

./logstash

5. php操作es

5.1 安装composer

https://www.runoob.com/w3cnote/composer-install-and-usage.html
Elasticsearch-php
tip：
Elasticsearch-php 的安装需要满足以下 4 个需求

PHP 7.0.0 或更高版本
Composer
ext-curl：PHP 的 Libcurl 扩展
原生 JSON 扩展 (ext-json) 1.3.7或更高版本

[root@localhost local]# mkdir elasticsearch-php
[root@localhost local]# cd elasticsearch-php/

vim composer.json写入如下内容

{
    "require": {
        "elasticsearch/elasticsearch": "~6.0"
    }
}

执行命令安装composer扩展

[root@localhost elasticsearch-php]# composer install --no-dev

vim search.php写入如下内容测试

<?php
require 'vendor/autoload.php';

$hosts = [
    '192.168.33.13:9200' // ip和端口
];

$client = Elasticsearch\ClientBuilder::create()
                                ->setHosts($hosts)
                                ->build();

$params = [
    'index' => 'sdk_account_device_info',
    'type' => '_doc',
    "size" => 10,  //代表每次分页查询的条数
    "from" => 10,  //代表每次分页查询起点
    //'id' => 'hy1b9aa0a216050816343786939d36680aa687' // http://192.168.247.140:9200/test/test/5
    'body' => [
        'query' => [
            'match' => [
                'imei' => 'b59c16aef5acb153'
            ]
        ]
    ]
];


$response = $client->search($params);
var_dump($response);

执行查看效果：

[root@localhost elasticsearch-php]# php search.php
array(4) {
  ["took"]=>
  int(2)
  ["timed_out"]=>
  bool(false)
  ["_shards"]=>
  array(4) {
    ["total"]=>
    int(1)
    ["successful"]=>
    int(1)
    ["skipped"]=>
    int(0)
    ["failed"]=>
    int(0)
  }
  ["hits"]=>
  array(3) {
    ["total"]=>
    array(2) {
      ["value"]=>
      int(90)
      ["relation"]=>
      string(2) "eq"
    }
    ["max_score"]=>
    float(2.5499628)
    ["hits"]=>
    array(10) {
      [0]=>
      array(5) {
        ["_index"]=>
        string(23) "sdk_account_device_info"
        ["_type"]=>
        string(4) "_doc"
        ["_id"]=>
        string(38) "397716164989830981c75643ff4b4edb...."
        ["_score"]=>
        float(2.5499628)
        ["_source"]=>
        array(9) {
          ["@timestamp"]=>
          string(24) "2021-06-18T10:33:03.783Z"
          ["user_id"]=>
          string(38) "397716164989830981c75643ff4b4edb...."
          ["reg_time"]=>
          string(24) "2021-03-23T11:29:43.000Z"
          ["idfv"]=>
          string(0) ""
          ["@version"]=>
          string(1) "1"
          ["oaid"]=>
          string(64) "F362B61F3575BECB9E2...."
          ["idfa"]=>
          string(0) ""
          ["imei"]=>
          string(16) "b59c16aef5acb153"
        }
      }
      [1]=>
      array(5) {
        ["_index"]=>
        string(23) "sdk_account_device_info"
        ["_type"]=>
......

安全相关

Elastic 安全是非常重要的。没有这个我们的数据可以被任何的人进行访问，串改，删除。Elastic Stack 的安全是由 x-pack 所提供的。在 Elastic Stack 7.0 版本之前，这个是商用的版本，需要进行安装，并购买。从Elastic Stack 7.0之后，x-pack 都已经在发布版中，所以不需要进行安装。我们只需要进行配置就可以了。
vim config/elasticsearch.yml 追加如下内容

http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

修改密码

[es@localhost elasticsearch]# elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y

Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

由于主要是es和kibana 所以我设置了es密码为 es123pw; kibana密码为 kibana123pw,其中es的用户名默认是 elastic，kibana的用户名默认是 kibana_system，其他的系统暂时不知道作用，因此我都是给了123456的密码。

修改kibana配置如下：

#elasticsearch.username: "kibana_system"
elasticsearch.username: "kibana_system"
#elasticsearch.password: "pass"
elasticsearch.password: "kibana123pw"

保证操作正常，重启下es和kibana，再次访问http://192.168.33.13:9200/?pretty和http://192.168.33.13:5601将会要求输入密码。

7.12.0 es+kibana+logstash
1. 安装java环境【说是Elasticsearch 7.0 开始，内置了 Java 环境，反正也不麻烦，在装下...
logstash导入文件日志到es
环境： elasticsearch-7.12.0-linux-x86_64logstash-7.12.0-linu...
docekr 安装elasticsearch:7.12.0
1. es 安装 2.创建对应的映射文件目录 3. 修改配置文件让其他机器可以访问 3.1 增加跨域配置 /roo...
Elasticsearch 常用查询命令
安装环境及版本：系统：es 7.12.0 一图形化界面 dsl: Domain Specific Langua...
gitlab版本升级并docker化
写在前面：这两天把公司的gitlab服务从老版本（7.12.0，源码版本）升级到了最新版（8.14.3，dock...
CentOS7 Docker部署Kibana+ElasticSe
此处安装的是7.12.0版本拉取镜像启动容器 -d 后台启动 --net 访问同一宿主机 -e 环境变量 -p...

7.12.0 es+kibana+logstash

1. 安装java环境【说是Elasticsearch 7.0 开始，内置了 Java 环境，反正也不麻烦，在装下吧】

1.1 下载相应版本的压缩包

1.2 解压缩并移至相应目录

2. 安装elasticsearch

2.1 下载

2.2 解压并移动到指定目录

2.3 创建用户并修改文件所属用户用户组

2.4 启动es

配置外网访问

测试启动

3. 安装kibana

3.1下载

3.2 解压并移动到指定目录

3.3 配置Kibana，更多配置： kibana配置文件

3.4 启动

4. 安装logstash

4.1 下载

4.2 解压并移动到指定目录

4.3 配置并启动

多表同步

5. php操作es

5.1 安装composer

5.2 建一个目录作为下载目录

安全相关

相关文章

7.12.0 es+kibana+logstash

logstash导入文件日志到es

docekr 安装elasticsearch:7.12.0

Elasticsearch 常用查询命令

gitlab版本升级并docker化

CentOS7 Docker部署Kibana+ElasticSe

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读