elasticsearch开启用户名密码和 ssl

作者: dozenx | 来源:发表于2021-07-30 10:43 被阅读0次

1添加账号

groupadd es

useradd es -g es -p es

2文件夹授权

mkdir -p /usr/local/apps/elasticsearch

mkdir -p /srv/elasticsearch/log

mkdir -p /srv/elasticsearch/data

chown -R es:es /usr/local/apps/elasticsearch

chown -R es:es /srv/elasticsearch

cd /usr/local/apps/elasticsearch

su es

3开始安装

cd /usr/local/apps/elasticsearch/

下载包

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.12.1-linux-x86_64.tar.gz

(如果你已经有了的话直接拷贝过来elasticsearch-7.12.1-linux-x86_64.tar.gz)

解压

tar -xvf elasticsearch-7.12.1-linux-x86_64.tar.gz

将文件移到目录下

将安装包复制到其他节点上

rsync elasticsearch-7.12.1-linux-x86_64.tar.gz rd@172.20.3.36:/usr/local/apps/elasticsearch/

rsync elasticsearch-7.12.1-linux-x86_64.tar.gz rd@172.20.3.12:/usr/local/apps/elasticsearch/

[rd@localhost elasticsearch] $cd elasticsearch-7.12.1/ [rd@localhost elasticsearch-7.12.1]$ mv * ../

删除空的文件夹

rm -rf elasticsearch-7.12.1

<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n4533" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">如果服务器的内存不够用的话需要修改内存大小

vi bin/elasticsearch

找到xms项目

ES_JAVA_OPTS="-Xms500m -Xmx500m"

启动

Option Description </pre>

-E <KeyValuePair> Configure a setting

-V, --version Prints Elasticsearch version information and exits

-d, --daemonize Starts Elasticsearch in the background

-h, --help Show help

-p, --pidfile <Path> Creates a pid file in the specified path on start

-q, --quiet Turns off standard output/error streams logging in console

-s, --silent Show minimal output

-v, --verbose Show verbose output

我们选择后台线程模式

创建用户

修改集群配置

vi config/elasticsearch.yml

cluster.name: search-center-es-cluster

(每台机器不同 36是 slave-node-1 12 是slave-node-2 35是 node.name: master-node-1)

node.name: slave-node-1

从机是false

node.master: true

node.data: true

discovery.zen.minimum_master_nodes: 2

discovery.zen.ping_timeout: 120s

bootstrap.system_call_filter: false

path.data: /srv/elasticsearch/data

path.logs: /srv/elasticsearch/logs

bootstrap.memory_lock: true

改成对应的ip地址

network.host: 172.20.3.35

http.port: 9200

discovery.seed_hosts: ["172.20.3.35:9300","172.20.3.12:9300"]

cluster.initial_master_nodes: ["master-node-1","master-node-2"]

认证配置

xpack.security.enabled: true

xpack.license.self_generated.type: basic

xpack.security.transport.ssl.enabled: true

xpack.security.transport.ssl.verification_mode: certificate

xpack.security.transport.ssl.client_authentication: required

xpack.security.transport.ssl.keystore.path: ssl/elastic-certificates.p12

xpack.security.transport.ssl.truststore.path: ssl/elastic-certificates.p12

xpack.security.http.ssl.enabled: true

xpack.security.http.ssl.keystore.path: http.p12

删除并创建es 的data 和log 目录

如果有必要rm -rf /srv/elasticsearch/data/*

如果有必要rm -rf /srv/elasticsearch/log/*

mkdir -p /srv/elasticsearch/data/

mkdir -p /srv/elasticsearch/log/

生成ca证书

bin/elasticsearch-certutil ca

选择默认路径文件名填写密码

生成凭证

bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

选择默认路径文件名填写密码

分发到集群的其他机器上

cp elastic-certificates.p12 config/ssl/

rsync config/ssl/elastic-certificates.p12 rd@172.20.3.12:/usr/local/apps/elasticsearch/config/ssl/

r4g9tj2z

rsync config/ssl/elastic-certificates.p12 rd@172.20.3.36:/usr/local/apps/elasticsearch/config/ssl/

r4g9tj2z

每台机器都得单独执行以下操作(注意下面的这一步如果错误的话是不会报错的小心输入你的密码保证和主机上输入的一致)

bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password

bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

bin/elasticsearch-keystore removexpack.security.transport.ssl.keystore.secure_password

bin/elasticsearch-keystore remove xpack.security.transport.ssl.truststore.secure_password

打开系统设置限制

报错 max file size [66920448] for user [rd] is too low, increase to [unlimited]

sudo sh -c "ulimit -n 65535 && exec su $LOGNAME"

sudo sh -c " ulimit -f unlimited "

启动每个服务

bin/elasticsearch -d

创建密码

bin/elasticsearch-setup-passwords auto // 自动随机生成并设置密码

Changed password for user apm_system PASSWORD apm_system = mzhAWtOQ2gWqRxsWHsRW

Changed password for user kibana_system PASSWORD kibana_system = WbfKBnMBpFhSmDfpshco

Changed password for user kibana PASSWORD kibana = WbfKBnMBpFhSmDfpshco

Changed password for user logstash_system PASSWORD logstash_system = 3v0TSw37tRCKKT89XXlP

Changed password for user beats_system PASSWORD beats_system = AgwwYr0cb0JcPFV7cDl4

Changed password for user remote_monitoring_user PASSWORD remote_monitoring_user = KrOzZf3FRP5csQuRfru1

Changed password for user elastic PASSWORD elastic = i0INDDaOpldg0Bk4TP9h

Changed password for user apm_system PASSWORD apm_system = QdXAj1VCFxr6sScc2lNK

Changed password for user kibana_system PASSWORD kibana_system = SoersS5DIx8Z5endzk6l

Changed password for user kibana PASSWORD kibana = SoersS5DIx8Z5endzk6l

Changed password for user logstash_system PASSWORD logstash_system = RiFofyTXxeysjJvW7qkM

Changed password for user beats_system PASSWORD beats_system = UXloMywJOQ61fpyzjCLm

Changed password for user remote_monitoring_user PASSWORD remote_monitoring_user = EYkFXKt9ZrovbJUOtVFX

Changed password for user elastic PASSWORD elastic = tb355XlypevZWw7I9L35:

测试密码是否都生效了

[rd@localhost elasticsearch]$ curl -u elastic http://172.20.3.35:9200/ Enter host password for user 'elastic': tb355XlypevZWw7I9L35 { "name" : "master-node-1", "cluster_name" : "search-center-es-cluster", "cluster_uuid" : "7JdDyc5PQa61YsUV64JA0w", "version" : { "number" : "7.12.1", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "3186837139b9c6b6d23c3200870651f10d3343b7", "build_date" : "2021-04-20T20:56:39.040728659Z", "build_snapshot" : false, "lucene_version" : "8.8.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" }

curl -u elastic http://172.20.3.36:9200/

<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n4644" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"> curl -u elastic http://172.20.3.36:
Enter host password for user 'elastic':
{
"name" : "slave-node-1",
"cluster_name" : "search-center-es-cluster",
"cluster_uuid" : "7JdDyc5PQa61YsUV64JA0w",
"version" : {
"number" : "7.12.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "3186837139b9c6b6d23c3200870651f10d3343b7",
"build_date" : "2021-04-20T20:56:39.040728659Z",
"build_snapshot" : false,
"lucene_version" : "8.8.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}

</pre>

curl -u elastic http://172.20.3.12:9200/

生成http证书 1:38(前面的一次失败了)

停止所有的es服务

bin/elasticsearch-certutil http

第一个选择N

第二个ca 选择y

/usr/local/apps/elasticsearch/elastic-stack-ca.p12

Generate a CSR? [y/N]n

Use an existing CA? [y/N]y

For how long should your certificate be valid? [5y]5y

Enter all the IP addresses that you need, one per line.

When you are done, press <ENTER> once more to move on to the next step.

172.20.3.35

172.20.3.12

172.20.3.36

You entered the following IP addresses.

172.20.3.35
172.20.3.12
172.20.3.36

Is this correct [Y/n]y

Do you wish to change any of these options? [y/N]n

Provide a password for the "http.p12" file: [<ENTER> for none]

What filename should be used for the output zip file? [/usr/local/apps/elasticsearch1/elasticsearch-ssl-http.zip]

/usr/local/apps/elasticsearch/elasticsearch-ssl-http2.zip

unzip elasticsearch-ssl-http.zip

[rd@localhost elasticsearch1]$ unzip elasticsearch-ssl-http.zip

Archive: elasticsearch-ssl-http.zip

creating: elasticsearch/

inflating: elasticsearch/README.txt

inflating: elasticsearch/http.p12

inflating: elasticsearch/sample-elasticsearch.yml

creating: kibana/

inflating: kibana/README.txt

inflating: kibana/elasticsearch-ca.pem

inflating: kibana/sample-kibana.yml

cp elasticsearch/http.p12 config/

分发https证书和凭证文件

rsync elasticsearch/http.p12 rd@172.20.3.12:/usr/local/apps/elasticsearch/config/

r4g9tj2z

rsync elasticsearch/http.p12 rd@172.20.3.36:/usr/local/apps/elasticsearch/config/

r4g9tj2z

每台机器执行以下命令

bin/elasticsearch-keystore remove xpack.security.transport.ssl.keystore.secure_password

bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password

bin/elasticsearch-keystore remove xpack.security.transport.ssl.truststore.secure_password

bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

bin/elasticsearch-keystore remove xpack.security.http.ssl.keystore.secure_password

bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password

修改配置文件

打开http.p12部分最后两行

测试使用curl 命令测试https 接口

openssl pkcs12 -in http.p12 -out client.pem -nokeys

openssl pkcs12 -in http.p12 -out key.pem -nocerts -nodes

curl -k --cert client.pem --key key.pem -u elastic https://192.168.211.117:9200/

curl -k --cert client.pem --key key.pem -u elastic https://1172.20.3.35:9200/

tb355XlypevZWw7I9L35

<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n4706" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">curl -k --cert client.pem --key key.pem -u elastic https://192.168.211.117:9200/
curl: /lib64/libcrypto.so.10: no version information available (required by /lib64/libssh2.so.1)
curl: /lib64/libcrypto.so.10: no version information available (required by /lib64/libldap-2.4.so.2)
curl: /lib64/libcrypto.so.10: no version information available (required by /lib64/libldap-2.4.so.2)
curl: /lib64/libssl.so.10: no version information available (required by /lib64/libldap-2.4.so.2)
Enter host password for user 'elastic':
{
"name" : "master-node-1", wqs@172.30.1.230
"cluster_name" : "search-center-es-cluster",
"cluster_uuid" : "7JdDyc5PQa61YsUV64JA0w",
"version" : {
"number" : "7.12.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "3186837139b9c6b6d23c3200870651f10d3343b7",
"build_date" : "2021-04-20T20:56:39.040728659Z",
"build_snapshot" : false,
"lucene_version" : "8.8.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
</pre>

curl -u elastic http://172.20.3.12:9201

测试ok 即可

使用java 客户端 restHeighLevel

将整个文件打压缩包投放到每个节点上

tar cvf elasticsearch1.tar elasticsearch1

rsync elasticsearch1.tar rd@172.20.3.36:/usr/local/apps/

rsync elasticsearch1.tar rd@172.20.3.12:/usr/local/apps/

r4g9tj2z

rsync elasticsearch/http.p12 rd@172.20.3.12:/usr/local/apps/elasticsearch1/config/

rsync config/http.p12 rd@172.20.3.12:/usr/local/apps/elasticsearch/config/

bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password

bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password

日志查看

tail -fn 200 /srv/elasticsearch/logs/search-center-es-cluster.log

最常见问题是主机启动正常了但是其余两个拷贝过去的服务器启动失败

ElasticsearchSecurityException[failed to load SSL configuration [xpack.security.transport.ssl]]; nested: ElasticsearchException[failed to initialize SSL TrustManager]; nested: IOException[parseAlgParameters failed: ObjectIdentifier() -- data isn't an object ID (tag = 48)]; nested: IOException[ObjectIdentifier() -- data isn't an object ID (tag = 48)];

Likely root cause: java.io.IOException: ObjectIdentifier() -- data isn't an object ID (tag = 48)

at sun.security.util.ObjectIdentifier.<init>(ObjectIdentifier.java:285)

at sun.security.util.DerInputStream.getOID(DerInputStream.java:321)

at com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:267)

at java.security.AlgorithmParameters.init(AlgorithmParameters.java:293)

at sun.security.pkcs12.PKCS12KeyStore.parseAlgParameters(PKCS12KeyStore.java:815)

at sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2027)

at java.security.KeyStore.load(KeyStore.java:1445)

at org.elasticsearch.xpack.core.ssl.TrustConfig.getStore(TrustConfig.java:98)

at org.elasticsearch.xpack.core.ssl.StoreTrustConfig.createTrustManager(StoreTrustConfig.java:66)

at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:439)

at java.util.HashMap.computeIfAbsent(HashMap.java:1127)

at org.elasticsearch.xpack.core.ssl.SSLService.lambda $loadSSLConfigurations$ 5(SSLService.java:528)

at java.util.HashMap.forEach(HashMap.java:1289)

at java.util.Collections$UnmodifiableMap.forEach(Collections.java:1507)

at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:526)

at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:144)

at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:462)

at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:292)

at org.elasticsearch.node.Node.lambda $new$ 17(Node.java:567)

at java.util.stream.ReferencePipeline $7$ 1.accept(ReferencePipeline.java:267)

at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)

at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)

at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)

at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)

at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)

at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)

at org.elasticsearch.node.Node.<init>(Node.java:571)

at org.elasticsearch.node.Node.<init>(Node.java:278)

at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:217)

at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:217)

<<<truncated>>>

For complete error details, refer to the log at /srv/elasticsearch/logs/search-center-es-cluster.log

启动服务

./elasticsearch -d

加密码

[elastic@console bin]$ ./elasticsearch-setup-passwords interactive

future versions of Elasticsearch will require Java 11; your Java version from [/usr/java/jdk1.8.0_181/jre] does not meet this requirement

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.

You will be prompted to enter passwords as the process progresses.

Please confirm that you would like to continue [y/N]y

Enter password for [elastic]:

Reenter password for [elastic]:

Passwords do not match.

Try again.

Enter password for [elastic]:

Reenter password for [elastic]:

Enter password for [apm_system]:

Reenter password for [apm_system]:

Enter password for [kibana]:

Reenter password for [kibana]:

Enter password for [logstash_system]:

Reenter password for [logstash_system]:

Enter password for [beats_system]:

Reenter password for [beats_system]:

Enter password for [remote_monitoring_user]:

Reenter password for [remote_monitoring_user]:

Changed password for user [apm_system]

Changed password for user [kibana]

Changed password for user [logstash_system]

Changed password for user [beats_system]

Changed password for user [remote_monitoring_user]

Changed password for user [elastic]

设置日志级别

机器学习和 x pack不支持 arm

ElasticsearchException[X-Pack is not supported and Machine Learning is not available for [linux-arm]; you can use the other X-Pack features (unsupported) by setting xpack.ml.enabled: false in elasticsearch.yml]

at org.elasticsearch.xpack.ml.MachineLearningFeatureSet.isRunningOnMlPlatform(MachineLearningFeatureSet.java:125)

at org.elasticsearch.xpack.ml.MachineLearningFeatureSet.isRunningOnMlPlatform(MachineLearningFeatureSet.java:116)

at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:666)

at org.elasticsearch.node.Node.lambda $new$ 17(Node.java:567)

at java.base/java.util.stream.ReferencePipeline $7$ 1.accept(ReferencePipeline.java:271)

at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)

at jav

xpack.ml.enabled: false

修改日志目录

path.data: /srv/elasticsearch/data

Path to log files:

path.logs: /srv/elasticsearch/log

network 标记为0.0.0.0

es@awifi

启动报错

ERROR: [3] bootstrap checks failed. You must address the points described in the following [3] lines before starting Elasticsearch.

bootstrap check failure [1] of [3]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]

bootstrap check failure [2] of [3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

bootstrap check failure [3] of [3]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

ERROR: Elasticsearch did not exit normally - check the logs at /srv/elasticsearch/log/elasticsearch.log

编辑配置

vi /etc/security/limits.conf

es soft nofile 65535

es hard nofile 65537

max file descriptors [4096] for elasticsearch process is too low,

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

elasticsearch启动时遇到的错误

问题翻译过来就是：elasticsearch用户拥有的内存权限太小，至少需要262144；

解决：

切换到root用户

执行命令：

sysctl -w vm.max_map_count=262144

查看结果：

sysctl -a|grep vm.max_map_count

显示：

vm.max_map_count = 262144

上述方法修改之后，如果重启虚拟机将失效，所以：

解决办法：

在 /etc/sysctl.conf文件最后添加一行

vm.max_map_count=262144

即可永久修改

ERROR: [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.

bootstrap check failure [1] of [1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

ERROR: Elasticsearch did not exit normally - check the logs at /srv/elasticsearch/log/elasticsearch.log

ps -ef | grep elasticsearch

查看是否启动正常

/usr/local/apps/elasticsearch/bin/elasticsearch 启动服务

bootstrap check failure [2] of [2]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

修改配置

vi /usr/local/apps/elasticsearch/conf/elasticsearch.yml

放开 cluster.initial_master_nodes: ["node-1", "node-2"]

ERROR: [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.

bootstrap check failure [1] of [1]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

解决：

Centos6不支持SecComp，而ES5.2.0默认bootstrap.system_call_filter为true

禁用：在elasticsearch.yml中配置bootstrap.system_call_filter为false，注意要在Memory下面:

bootstrap.memory_lock: false

bootstrap.system_call_filter: false

终于启动成功了

添加 ssl 和用户名密码

在elasticsearch 主目录下bin下面执行命令

elasticsearch-certgen

Let's get started...

Please enter the desired output file [certificate-bundle.zip]: cert.zip

[图片上传失败...(image-56e3da-1627612953435)]

Please enter the desired output file [certificate-bundle.zip]: cert.zip (最终生成文件的位置)

Enter instance name: bigdata

Enter name for directories and files [bigdata]: bigdata

Enter IP Addresses for instance (comma-separated if more than one) []: (ip地址多个用逗号分割)192.168.211.117,192.168.211.118,192.168.211.119

Enter DNS names for instance (comma-separated if more than one) []: 192.168.211.117,192.168.211.118,192.168.211.119

Would you like to specify another instance? Press 'y' to continue entering instance information: n

Certificates written to /usr/local/apps/elasticsearch/elasticsearch-7.12.1/cert.zip (这里是告诉你生成的位置)

This file should be properly secured as it contains the private keys for all

instances and the certificate authority.

这里的enter instance name 的作用是

dns names for instance

启动elasticsearch 报错: 看来6.几的版本和7.几的版本不一样

rd@hadoop-server-001 bin]$ uncaught exception in thread [main]

java.lang.IllegalArgumentException: unknown setting [xpack.ssl.key] did you mean [xpack.http.ssl.key]?

at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:533)

at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:478)

at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:449)

at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:420)

at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:138)

at org.elasticsearch.node.Node.<init>(Node.java:396)

at org.elasticsearch.node.Node.<init>(Node.java:278)

at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:217)

at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:217)

at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:397)

at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)

at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)

at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75)

at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)

at org.elasticsearch.cli.Command.main(Command.java:79)

at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115)

at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81)

For complete error details, refer to the log at /usr/local/apps/elasticsearch/elasticsearch-7.12.1/logs/elasticsearch.log

https://elasticstack.blog.csdn.net/article/details/105044365

[3] bootstrap checks failed. You must address the points described in the following [3] lines before starting Elasticsearch.

bootstrap check failure [1] of [3]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]

需要用root 执行以下命令

ulimit -n 65535

如果是非root 用户需要退出重新登录

[2] bootstrap checks failed. You must address the points described in the following [2] lines before starting Elasticsearch.

bootstrap check failure [1] of [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

ERROR: [2] bootstrap checks failed. You must address the points described in the following [2] lines before starting Elasticsearch.

bootstrap check failure [1] of [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

ERROR: Elasticsearch did not exit normally - check the logs at /usr/local/apps/elasticsearch/elasticsearch-7.12.1/logs/elasticsearch.log

elasticsearch启动时遇到的错误

问题翻译过来就是：elasticsearch用户拥有的内存权限太小，至少需要262144；

解决：

切换到root用户

执行命令：

sysctl -w vm.max_map_count=262144

查看结果：

sysctl -a|grep vm.max_map_count

显示：

vm.max_map_count = 262144

上述方法修改之后，如果重启虚拟机将失效，所以：

解决办法：

在 /etc/sysctl.conf文件最后添加一行

vm.max_map_count=262144

即可永久修改

修改elasticsearch.yml 找到discovery那一块，做如下修改

cluster.initial_master_nodes: ["node-1","node-2"]修改为：cluster.initial_master_nodes: ["node-1"]

sudo sh -c "ulimit -n 65535 && exec su $LOGNAME"

建表语句

collect_trunk_flow

<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n4940" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">curl -k --cert client.pem --key key.pem -u elastic -X PUT 'https://192.168.211.117:9200/collect_trunk_flow' \

--header 'Content-Type:application/json' \

--data '{

"settings": {

"number_of_shards": 5,

"number_of_replicas": 1

"mappings": {

"properties": {

"id": {

"type": "long"

"nfid": {

"type": "keyword"

"timetick": {

"type": "long"

"account": {

"type": "keyword"

"ipv4_wan_status": {

"type": "keyword"

"ipv4_lan_status": {

"type": "keyword"

"ipv6_wan_status": {

"type": "keyword"

"ipv6_lan_status": {

"type": "keyword"

"pppoe_current_err": {

"type": "integer"

"up_bytes": {

"type": "long"

"dn_bytes": {

"type": "long"

"up_pkts": {

"type": "long"

"dn_pkts": {

"type": "long"

"resource": {

"type": "keyword"

"prefer_time": {

"type": "long"

"valid_time": {

"type": "long"

"create_time": {

"type": "long"

"update_time": {

"type": "long"

"ipv4_wan_ip": {

"type": "keyword"

"ipv6_wan_ip": {

"type": "keyword"

"pppoe_first_err": {

"type": "integer"

"pppoe_first_err_time": {

"type": "long"

"pppoe_session": {

"type": "keyword"

"up_bps": {

"type": "long"

"dn_bps": {

"type": "long"

"up_pkts_add": {

"type": "long"

"dn_pkts_add": {

"type": "long"

"up_bytes_add": {

"type": "long"

"dn_bytes_add": {

"type": "long"

"pppoe_last_dial_time": {

"type": "long"

}

</pre>

collect_tunnel_flow

curl -k --cert client.pem --key key.pem -u elastic -X PUT 'https://192.168.211.117:9200/collect_tunnel_flow' \ --header 'Content-Type:application/json' \ --data '{

"settings": {

"number_of_shards": 3,

"number_of_replicas": 1

"mappings": {

"properties": {

"rowkey": {

<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n4952" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">"type": "keyword"</pre>

"nfid": {

<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n4955" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">"type": "keyword"</pre>

"type": {

<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n4958" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">"type": "keyword"</pre>

"timetick": {

"recv_pkt": {

"recv_byte": {

"recv_drop_pkt": {

"send_pkt": {

"send_byte": {

"send_drop_pkt": {

"last_send_drop_pkt": {

"last_send_byte": {

"last_send_pkt": {

"last_recv_drop_pkt": {

"last_recv_byte": {

"last_recv_pkt": {

"create_time": {

}

curl -k --cert client.pem --key key.pem -u elastic -X PUT 'https://192.168.211.117:9200/collect_trunk_flow' \ --header 'Content-Type:application/json' \ --data '{ "settings": { "number_of_shards": 5, "number_of_replicas": 1 }, "mappings": { "properties": { "id": { "type": "long" }, "nfid": { "type": "keyword" }, "timetick": { "type": "long" }, "account": { "type": "keyword" }, "ipv4_wan_status": { "type": "keyword" }, "ipv4_lan_status": { "type": "keyword" }, "ipv6_wan_status": { "type": "keyword" }, "ipv6_lan_status": { "type": "keyword" }, "pppoe_current_err": { "type": "integer" }, "up_bytes": { "type": "long" }, "dn_bytes": { "type": "long" }, "up_pkts": { "type": "long" }, "dn_pkts": { "type": "long" }, "resource": { "type": "keyword" }, "prefer_time": { "type": "long" }, "valid_time": { "type": "long" }, "create_time": { "type": "long" }, "update_time": { "type": "long" }, "ipv4_wan_ip": { "type": "keyword" }, "ipv6_wan_ip": { "type": "keyword" }, "pppoe_first_err": { "type": "integer" }, "pppoe_first_err_time": { "type": "long" }, "pppoe_session": { "type": "keyword" }, "up_bps": { "type": "long" }, "dn_bps": { "type": "long" }, "up_pkts_add": { "type": "long" }, "dn_pkts_add": { "type": "long" }, "up_bytes_add": { "type": "long" }, "dn_bytes_add": { "type": "long" }, "pppoe_last_dial_time": { "type": "long" } } } }'

curl -k --cert client.pem --key key.pem -u elastic -X DELETE 'https://192.168.211.117:9200/collect_tunnel_flow' \ --header 'Content-Type:application/json'

curl -k --cert client.pem --key key.pem -u elastic -X delete 'https://192.168.211.117:9200/collect_tunnel_flow' \

collect_subintf_flow

curl -k --cert client.pem --key key.pem -u elastic -X PUT 'https://192.168.211.117:9200/collect_tunnel_flow' \ --header 'Content-Type:application/json' \ --data '{ "settings": { "number_of_shards": 3, "number_of_replicas": 1 }, "mappings": { "properties": { "rowkey": { "type": "keyword" }, "nfid": { "type": "keyword" }, "type": { "type": "keyword" }, "timetick": { "type": "long" }, "recv_pkt": { "type": "long" }, "recv_byte": { "type": "long" }, "recv_drop_pkt": { "type": "long" }, "send_pkt": { "type": "long" }, "send_byte": { "type": "long" }, "send_drop_pkt": { "type": "long" }, "last_send_drop_pkt": { "type": "long" }, "last_send_byte": { "type": "long" }, "last_send_pkt": { "type": "long" }, "last_recv_drop_pkt": { "type": "long" }, "last_recv_byte": { "type": "long" }, "last_recv_pkt": { "type": "long" }, "create_time": { "type": "long" } } } }'

curl -k --cert client.pem --key key.pem -u elastic -X DELETE 'https://192.168.211.117:9200/collect_terminal_info' \ --header 'Content-Type:application/json'

collect_terminal_info

curl -k --cert client.pem --key key.pem -u elastic -X PUT 'https://192.168.211.117:9200/collect_terminal_info' \ --header 'Content-Type:application/json' \ --data '{ "settings": { "number_of_shards": 3, "number_of_replicas": 1 }, "mappings": { "properties": { "rowkey": { "type": "keyword" }, "nfid": { "type": "keyword" }, "timetick": { "type": "long" }, "account": { "type": "keyword" }, "mac": { "type": "keyword" }, "ip": { "type": "keyword" }, "gw": { "type": "keyword" }, "mask": { "type": "keyword" }, "ipv6": { "type": "keyword" }, "ipv6_gw": { "type": "keyword" }, "link_local": { "type": "keyword" }, "desc": { "type": "keyword" }, "status": { "type": "keyword" }, "up_bytes": { "type": "long" }, "dn_bytes": { "type": "long" }, "up_pkts": { "type": "long" }, "dn_pkts": { "type": "long" }, "create_time": { "type": "long" }, "up_bps": { "type": "long" }, "dn_bps": { "type": "long" }, "up_pkts_add": { "type": "long" }, "dn_pkts_add": { "type": "long" }, "up_bytes_add": { "type": "long" }, "dn_bytes_add": { "type": "long" } } } }'

curl -k --cert client.pem --key key.pem -u elastic -X DELETE 'https://192.168.211.117:9200/collect_vport_info' \ --header 'Content-Type:application/json'

collect_vport_info

curl -k --cert client.pem --key key.pem -u elastic -X PUT 'https://192.168.211.117:9200/collect_vport_info' \ --header 'Content-Type:application/json' \ --data '{ "settings": { "number_of_shards": 5, "number_of_replicas": 1 }, "mappings": { "properties": { "id": { "type": "long" }, "nfid": { "type": "keyword" }, "vcpe_nfid": { "type": "keyword" }, "type": { "type": "keyword" }, "timetick": { "type": "long" }, "recv_pkt": { "type": "long" }, "recv_byte": { "type": "long" }, "recv_drop_pkt": { "type": "long" }, "send_pkt": { "type": "long" }, "send_byte": { "type": "long" }, "send_drop_pkt": { "type": "long" }, "last_send_drop_pkt": { "type": "long" }, "last_send_byte": { "type": "long" }, "last_send_pkt": { "type": "long" }, "last_recv_drop_pkt": { "type": "long" }, "last_recv_byte": { "type": "long" }, "last_recv_pkt": { "type": "long" }, "create_time": { "type": "long" }, "update_time": { "type": "long" }, "online_user_num": { "type": "integer" }, "offline_user_num": { "type": "integer" }, "open_user_num": { "type": "integer" } } } }'

curl -k --cert client.pem --key key.pem -u elastic -X DELETE 'https://192.168.211.117:9200/collect_vcpe_status' \ --header 'Content-Type:application/json'

collect_vcpe_status

curl -k --cert client.pem --key key.pem -u elastic -X PUT 'https://192.168.211.117:9200/collect_vcpe_status' \ --header 'Content-Type:application/json' \ --data '{ "settings": { "number_of_shards": 5, "number_of_replicas": 1 }, "mappings": { "properties": { "id": { "type": "long" }, "nfid": { "type": "keyword" }, "timetick": { "type": "long" }, "account": { "type": "keyword" }, "ipv4_wan_status": { "type": "keyword" }, "ipv4_lan_status": { "type": "keyword" }, "ipv6_wan_status": { "type": "keyword" }, "ipv6_lan_status": { "type": "keyword" }, "pppoe_current_err": { "type": "integer" }, "up_bytes": { "type": "long" }, "dn_bytes": { "type": "long" }, "up_pkts": { "type": "long" }, "dn_pkts": { "type": "long" }, "resource": { "type": "keyword" }, "prefer_time": { "type": "long" }, "valid_time": { "type": "long" }, "create_time": { "type": "long" }, "update_time": { "type": "long" }, "ipv4_wan_ip": { "type": "keyword" }, "ipv6_wan_ip": { "type": "keyword" }, "pppoe_first_err": { "type": "integer" }, "pppoe_first_err_time": { "type": "long" }, "pppoe_session": { "type": "keyword" }, "up_bps": { "type": "long" }, "dn_bps": { "type": "long" }, "up_pkts_add": { "type": "long" }, "dn_pkts_add": { "type": "long" }, "up_bytes_add": { "type": "long" }, "dn_bytes_add": { "type": "long" }, "pppoe_last_dial_time": { "type": "long" } } } }'

collect_server_hardware_status

curl -k --cert client.pem --key key.pem -u elastic -X PUT 'https://192.168.211.117:9200/collect_server_hardware_status' \ --header 'Content-Type:application/json' \ --data '{ "settings": { "number_of_shards": 5, "number_of_replicas": 1 }, "mappings": { "properties": { "id": { "type": "long" }, "ip": { "type": "keyword" }, "timetick": { "type": "long" }, "area_code": { "type": "keyword" }, "max_cpu": { "type": "keyword" }, "max_cpu_info": { "type": "keyword" }, "memory_rate": { "type": "keyword" }, "disk_rate": { "type": "keyword" }, "max_temperature": { "type": "keyword" }, "cpu": { "type": "keyword" }, "memory": { "type": "keyword" }, "disk": { "type": "keyword" }, "temperature": { "type": "keyword" }, "create_time": { "type": "long" }, "update_time": { "type": "long" } } } }'

collect_vni_flow

curl -k --cert client.pem --key key.pem -u elastic -X PUT 'https://192.168.211.117:9200/collect_vni_flow' \ --header 'Content-Type:application/json' \ --data '{ "settings": { "number_of_shards": 5, "number_of_replicas": 1 }, "mappings": { "properties": { "id": { "type": "long" }, "nfid": { "type": "keyword" }, "type": { "type": "keyword" }, "timetick": { "type": "long" }, "recv_pkt": { "type": "long" }, "recv_byte": { "type": "long" }, "recv_drop_pkt": { "type": "long" }, "send_pkt": { "type": "long" }, "send_byte": { "type": "long" }, "send_drop_pkt": { "type": "long" }, "last_send_err_pkt": { "type": "long" }, "last_recv_err_pkt": { "type": "long" }, "last_send_drop_pkt": { "type": "long" }, "last_send_byte": { "type": "long" }, "last_send_pkt": { "type": "long" }, "last_recv_drop_pkt": { "type": "long" }, "last_recv_byte": { "type": "long" }, "last_recv_pkt": { "type": "long" }, "create_time": { "type": "long" }, "update_time": { "type": "long" }, "vni": { "type": "integer" }, "pvlan": { "type": "integer" }, "cvlan": { "type": "integer" }, "total_terminal": { "type": "integer" }, "user_type": { "type": "integer" }, "recv_bps": { "type": "long" }, "recv_byte_add": { "type": "long" }, "send_bps": { "type": "long" }, "send_pkt_add": { "type": "long" }, "send_byte_add": { "type": "long" }, "recv_pkt_add": { "type": "long" } } } }'

elasticsearch开启用户名密码和 ssl

1添加账号

3开始安装

创建用户

修改集群配置

(每台机器不同 36是 slave-node-1 12 是slave-node-2 35是 node.name: master-node-1)

从机是false

改成对应的ip地址

认证配置

xpack.security.http.ssl.enabled: true

xpack.security.http.ssl.keystore.path: http.p12

生成ca证书

生成凭证

启动每个服务

创建密码

生成http证书 1:38(前面的一次失败了)

每台机器执行以下命令

测试使用curl 命令测试https 接口

将整个文件打压缩包投放到每个节点上

日志查看

Path to log files:

sudo sh -c "ulimit -n 65535 && exec su $LOGNAME"