最近demo 3.0环境,查看CPU始终显示ES的能消耗100%的CPU
在平台操作,比如打开API门户也是发现显示列表会变得很慢。
本想通过 jvmtop.sh 找出java、进程中那个cpu使用率这么高的线程在干嘛, 结果 jstack 版本低于启动ES使用的JVM版本!
网上浏览了下别人的博客对各类问题的定位,注意到一个GC过于频繁可能会让ES呈现出上述表现
于是查看ES的日志:
cd /data/elasticsearch/log
摘录如下部分日志:
[2020-06-23T12:09:36,903][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361338] overhead, spent [5.6s] collecting in the last [5.9s]
[2020-06-23T12:09:40,953][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361339] overhead, spent [4s] collecting in the last [4s]
[2020-06-23T12:09:47,327][INFO ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][old][24361340][358347] duration [6.3s], collections [1]/[6.3s], total [6.3s]/[13.7d], memory [1.8gb]->[1.8gb]/[1.9gb], all_pools {[young] [217.5mb]->[219.1mb]/[266.2mb]}{[survivor] [0b]->[0b]/[33.2mb]}{[old] [1.6gb]->[1.6gb]/[1.6gb]}
[2020-06-23T12:09:47,327][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361340] overhead, spent [6.3s] collecting in the last [6.3s]
[2020-06-23T12:09:51,716][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361341] overhead, spent [4.1s] collecting in the last [4.3s]
[2020-06-23T12:09:57,706][INFO ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][old][24361342][358349] duration [5.7s], collections [1]/[5.9s], total [5.7s]/[13.7d], memory [1.8gb]->[1.8gb]/[1.9gb], all_pools {[young] [210.6mb]->[220.2mb]/[266.2mb]}{[survivor] [0b]->[0b]/[33.2mb]}{[old] [1.6gb]->[1.6gb]/[1.6gb]}
[2020-06-23T12:09:57,706][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361342] overhead, spent [5.7s] collecting in the last [5.9s]
[2020-06-23T12:10:02,124][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361343] overhead, spent [4.2s] collecting in the last [4.4s]
[2020-06-23T12:10:08,361][INFO ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][old][24361344][358351] duration [5.8s], collections [1]/[6.2s], total [5.8s]/[13.7d], memory [1.8gb]->[1.8gb]/[1.9gb], all_pools {[young] [209.5mb]->[208.4mb]/[266.2mb]}{[survivor] [0b]->[0b]/[33.2mb]}{[old] [1.6gb]->[1.6gb]/[1.6gb]}
[2020-06-23T12:10:08,361][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361344] overhead, spent [5.8s] collecting in the last [6.2s]
[2020-06-23T12:10:13,650][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361346] overhead, spent [4.1s] collecting in the last [4.1s]
确实发现每隔几秒钟就GC一次!
看了默认2G的堆内存需要调高些了。
/etc/elasticsearch/jvm.options
-Xms3048M
-Xmx3048M
鉴于服务器空余内存不多,就先加1GB上去看看效果。
systemctl stop elasticsearch
systemctl start elasticsearch
重启后过3分钟看top -d 1
,ES 的CPU占用果然正常多了。
网友评论