这里我在虚拟机里面安装的,Ubuntu22.04,4核8G内存,要安装Elasticsearch集群+Kibana可视化工具。Elasticsearch的部署官网都是给了教程的。我们可以直接参考。
- 单节点安装的:https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_start_a_single_node_cluster
- 集群安装:https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker-compose-file
官网给的集群安装是在一个compose文件里面的,也就是在一个机器里面安装的集群,没有分在不同的节点。官网这么给的,我们就这么用。机器多给点内存和硬盘。
1. 修改系统内核文件
- 首先,需要修改
vm.max_map_count
的值,不然可能报错:max virtual memory areas vm.max_map_count [65530] is too low
。
# 1. 修改
vim /etc/sysctl.conf
# 在最后添加如下内容
vm.max_map_count=262144
# 2. 使改动生效
sysctl -p
2. 配置文件
官网给了 2个。注意这2个配置文件要在同一目录下。
.env
和docker-compose.yml
2.1 env
# Password for the 'elastic' user (at least 6 characters)
ELASTIC_PASSWORD=123456
# Password for the 'kibana_system' user (at least 6 characters)
KIBANA_PASSWORD=123abc
# Version of Elastic products
STACK_VERSION=8.14.3
# Set the cluster name
CLUSTER_NAME=docker-cluster
# Set to 'basic' or 'trial' to automatically start the 30-day trial
LICENSE=basic
#LICENSE=trial
# Port to expose Elasticsearch HTTP API to the host
ES_PORT=9200
#ES_PORT=127.0.0.1:9200
# Port to expose Kibana to the host
KIBANA_PORT=5601
#KIBANA_PORT=80
# Increase or decrease based on the available host memory (in bytes)
MEM_LIMIT=1073741824
# Project namespace (defaults to the current folder name if not set)
#COMPOSE_PROJECT_NAME=myproject
2.2 docker-compose.yml
这个文件建了5个service,因为elasticsearch要求要证书什么的,第一个是为了生成统一证书,好像还在修改了es01中kibana_system
账号的密码,以后就没用了;2、3、4都是elasticsearch服务,都是用的同一个证书;最后是kibana服务。
映射目录稍作了改动,加了网络和ip(注意它生成证书的时候指定了ip,那里也要改成指定的IP),其他基本和官网一样。
#version: '3.8'
services:
es-certs:
env_file:
- .env
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
container_name: es-certs
privileged: true
volumes:
- /opt/soft/elasticsearch/config/certs:/usr/share/elasticsearch/config/certs
user: "0"
command: >
bash -c '
if [ x${ELASTIC_PASSWORD} == x ]; then
echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
exit 1;
elif [ x${KIBANA_PASSWORD} == x ]; then
echo "Set the KIBANA_PASSWORD environment variable in the .env file";
exit 1;
fi;
if [ ! -f config/certs/ca.zip ]; then
echo "Creating CA";
bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
unzip config/certs/ca.zip -d config/certs;
fi;
if [ ! -f config/certs/certs.zip ]; then
echo "Creating certs";
echo -ne \
"instances:\n"\
" - name: es01\n"\
" dns:\n"\
" - es01\n"\
" - localhost\n"\
" ip:\n"\
" - 172.18.0.11\n"\
" - name: es02\n"\
" dns:\n"\
" - es02\n"\
" - localhost\n"\
" ip:\n"\
" - 172.18.0.12\n"\
" - name: es03\n"\
" dns:\n"\
" - es03\n"\
" - localhost\n"\
" ip:\n"\
" - 172.18.0.13\n"\
> config/certs/instances.yml;
bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
unzip config/certs/certs.zip -d config/certs;
fi;
echo "Setting file permissions"
chown -R root:root config/certs;
find . -type d -exec chmod 750 \{\} \;;
find . -type f -exec chmod 640 \{\} \;;
echo "Waiting for Elasticsearch availability";
until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
echo "Setting kibana_system password";
until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
echo "All done!";
'
healthcheck:
test: [ "CMD-SHELL", "[ -f config/certs/es01/es01.crt ]" ]
interval: 1s
timeout: 5s
retries: 120
networks:
elastic:
ipv4_address: 172.18.0.10
es01:
env_file:
- .env
depends_on:
es-certs:
condition: service_healthy
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
container_name: es01
hostname: es01
restart: always
privileged: true
volumes:
- /opt/soft/elasticsearch/config/certs:/usr/share/elasticsearch/config/certs
- '/opt/soft/elasticsearch/es01/plugins:/usr/share/elasticsearch/plugins'
- '/opt/soft/elasticsearch/es01/data:/usr/share/elasticsearch/data'
- '/opt/soft/elasticsearch/es01/logs:/usr/share/elasticsearch/logs'
ports:
- ${ES_PORT}:9200
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "TZ=Asia/Shanghai"
- "http.host=0.0.0.0"
- node.name=es01
- cluster.name=${CLUSTER_NAME}
# 选举主节点master资格的节点
- cluster.initial_master_nodes=es01,es02,es03
- discovery.seed_hosts=es02,es03
- ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
- bootstrap.memory_lock=true
# 默认为true,表示启用 Elasticsearch 安全功能
- xpack.security.enabled=true
# 用于在 Elasticsearch 用于与其他客户端通信的 HTTP 网络层上启用或禁用 TLS/SSL。默认值为false:
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/es01/es01.key
- xpack.security.http.ssl.certificate=certs/es01/es01.crt
- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
# 用于在传输网络层上启用或禁用 TLS/SSL,节点间相互通信。默认值为false
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/es01/es01.key
- xpack.security.transport.ssl.certificate=certs/es01/es01.crt
- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.license.self_generated.type=${LICENSE}
deploy:
resources:
limits:
memory: ${MEM_LIMIT}
# 句柄数配置
ulimits:
memlock:
soft: -1
hard: -1
#nofile:
# soft: 65536
#hard: 65536
healthcheck:
test:
[
"CMD-SHELL",
"curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
]
interval: 10s
timeout: 10s
retries: 120
networks:
elastic:
ipv4_address: 172.18.0.11
es02:
env_file:
- .env
depends_on:
- es01
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
privileged: true
container_name: es02
hostname: es02
restart: always
volumes:
- /opt/soft/elasticsearch/config/certs:/usr/share/elasticsearch/config/certs
- '/opt/soft/elasticsearch/es02/plugins:/usr/share/elasticsearch/plugins'
- '/opt/soft/elasticsearch/es02/data:/usr/share/elasticsearch/data'
- '/opt/soft/elasticsearch/es02/logs:/usr/share/elasticsearch/logs'
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "TZ=Asia/Shanghai"
- node.name=es02
- cluster.name=${CLUSTER_NAME}
- cluster.initial_master_nodes=es01,es02,es03
- discovery.seed_hosts=es01,es03
- bootstrap.memory_lock=true
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/es02/es02.key
- xpack.security.http.ssl.certificate=certs/es02/es02.crt
- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/es02/es02.key
- xpack.security.transport.ssl.certificate=certs/es02/es02.crt
- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.license.self_generated.type=${LICENSE}
deploy:
resources:
limits:
memory: ${MEM_LIMIT}
# 句柄数配置
ulimits:
memlock:
soft: -1
hard: -1
# nofile:
# soft: 65536
# hard: 65536
healthcheck:
test:
[
"CMD-SHELL",
"curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
]
interval: 10s
timeout: 10s
retries: 120
networks:
elastic:
ipv4_address: 172.18.0.12
es03:
env_file:
- .env
depends_on:
- es02
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
container_name: es03
hostname: es03
restart: always
privileged: true
volumes:
- /opt/soft/elasticsearch/config/certs:/usr/share/elasticsearch/config/certs
- '/opt/soft/elasticsearch/es03/plugins:/usr/share/elasticsearch/plugins'
- '/opt/soft/elasticsearch/es03/data:/usr/share/elasticsearch/data'
- '/opt/soft/elasticsearch/es03/logs:/usr/share/elasticsearch/logs'
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "TZ=Asia/Shanghai"
- node.name=es03
- cluster.name=${CLUSTER_NAME}
- cluster.initial_master_nodes=es01,es02,es03
- discovery.seed_hosts=es01,es02
- bootstrap.memory_lock=true
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/es03/es03.key
- xpack.security.http.ssl.certificate=certs/es03/es03.crt
- xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/es03/es03.key
- xpack.security.transport.ssl.certificate=certs/es03/es03.crt
- xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.license.self_generated.type=${LICENSE}
deploy:
resources:
limits:
memory: ${MEM_LIMIT}
# 句柄数配置
ulimits:
memlock:
soft: -1
hard: -1
#nofile:
# soft: 65536
# hard: 65536
healthcheck:
test:
[
"CMD-SHELL",
"curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
]
interval: 10s
timeout: 10s
retries: 120
networks:
elastic:
ipv4_address: 172.18.0.13
kibana:
env_file:
- .env
depends_on:
es01:
condition: service_healthy
es02:
condition: service_healthy
es03:
condition: service_healthy
image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
container_name: kibana
hostname: kibana
restart: always
privileged: true
volumes:
- /opt/soft/elasticsearch/config/certs:/usr/share/kibana/config/certs
- /opt/soft/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml
- '/opt/soft/kibana/data:/usr/share/kibana/data'
ports:
- ${KIBANA_PORT}:5601
environment:
- SERVERNAME=kibana
- ELASTICSEARCH_HOSTS=https://es01:9200
- ELASTICSEARCH_USERNAME=kibana_system
- ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
- ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
deploy:
resources:
limits:
memory: ${MEM_LIMIT}
healthcheck:
test:
[
"CMD-SHELL",
"curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
]
interval: 10s
timeout: 10s
retries: 120
networks:
elastic:
ipv4_address: 172.18.0.14
# 自定义网络 elastic
networks:
elastic:
# 启动时不自动创建,需要提前手动创建 docker network create -d bridge elastic
external: true
driver: bridge
# https://www.w3cschool.cn/doc_docker_1_11/docker_1_11-engine-reference-commandline-volume_create-index.html
# 创建的 volume 将存储到 /var/lib/docker/volumes/ 路径下
#volumes:
# CA 证书 挂载
# certs:
# driver: local
3. 运行compose文件
注意:
- 由于在国内,用到的镜像自己提前想办法下载下来,不然构建不成功的
- 文件中映射的目录和文件要提前创建好,并给予读写权限
- 上面用到了
kibana.yml
配置文件,主要来设置中文的,原来写在 environment底下了,但是没生效。
kibana.yml
server.host: "0.0.0.0"
server.shutdownTimeout: "5s"
# 连接es集群配置多个地址,单机一个地址
elasticsearch.hosts: ["http://10.10.1.31:9200"]
#elasticsearch.username: "test"
#elasticsearch.password: "zrb123"
# 设置kibana中文
i18n.locale: "zh-CN"
3.1 运行命令
# 1. 检查文件格式有没有问题
docker compose -f docker-compose-elastic.yml config -q
# 2. 运行
docker compose -f docker-compose-elastic.yml up -d
elastic.png
容器
可以看出es还是挺占内存的。
3.2 查看
先看看elasticsearch:https://10.10.1.31:9200
,账号 elastic
,密码就是配置文件中设置的。
再看看kibana:
http://10.10.1.31:5601
,账号密码 都用 上面elasticsearch的。kibana
4. 补充
其实,如果只是一主2从节点的话,官网的这个后期并不一定好维护,好多东西都写在了compose文件中,不好找了。
- 关于证书文件,我觉得自己的证书文件可以移到自己节点目录下,虽然都是一样的,生成完以后,复制到自己节点目录下,这样目录映射也是映射自己节点目录
- 关于一些配置,我更喜欢放到配置文件中,什么账号、密码、节点名称、集群名称,放到es的配置文件(应该是
config/elasticsearch.yml
)中,就像kibana.yml
那个文件一样,然后做映射。这样后期维护感觉更好用。
- 关于一些配置,我更喜欢放到配置文件中,什么账号、密码、节点名称、集群名称,放到es的配置文件(应该是
-
elasticsearch设置时常用的一些工具
bin目录
-
我们先来看一下它自带的账号有哪些:
可以看到有好几个,连接kibana和logstash的都有。其中
elastic
应该是管理员账号,密码是自己写在配置文件中的。上面第一个服务中 也给了通过接口修改其他账号密码的命令,可以参考。
添加账号:
# 添加test用户
./elasticsearch-users useradd test
# 授予超级管理员角色
./elasticsearch-users roles -a superuser test
# 授予kibana的用户角色
./elasticsearch-users roles -a kibana_system test
添加账号
网友评论