美文网首页
2020-06-22 记一次ES占用CPU异常高的问题处理

2020-06-22 记一次ES占用CPU异常高的问题处理

作者: 五大RobertWu伍洋 | 来源:发表于2020-06-24 11:09 被阅读0次

最近demo 3.0环境,查看CPU始终显示ES的能消耗100%的CPU

在平台操作,比如打开API门户也是发现显示列表会变得很慢。

本想通过 jvmtop.sh 找出java、进程中那个cpu使用率这么高的线程在干嘛, 结果 jstack 版本低于启动ES使用的JVM版本!

网上浏览了下别人的博客对各类问题的定位,注意到一个GC过于频繁可能会让ES呈现出上述表现

于是查看ES的日志:

cd /data/elasticsearch/log

摘录如下部分日志:

[2020-06-23T12:09:36,903][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361338] overhead, spent [5.6s] collecting in the last [5.9s]
[2020-06-23T12:09:40,953][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361339] overhead, spent [4s] collecting in the last [4s]
[2020-06-23T12:09:47,327][INFO ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][old][24361340][358347] duration [6.3s], collections [1]/[6.3s], total [6.3s]/[13.7d], memory [1.8gb]->[1.8gb]/[1.9gb], all_pools {[young] [217.5mb]->[219.1mb]/[266.2mb]}{[survivor] [0b]->[0b]/[33.2mb]}{[old] [1.6gb]->[1.6gb]/[1.6gb]}
[2020-06-23T12:09:47,327][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361340] overhead, spent [6.3s] collecting in the last [6.3s]
[2020-06-23T12:09:51,716][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361341] overhead, spent [4.1s] collecting in the last [4.3s]
[2020-06-23T12:09:57,706][INFO ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][old][24361342][358349] duration [5.7s], collections [1]/[5.9s], total [5.7s]/[13.7d], memory [1.8gb]->[1.8gb]/[1.9gb], all_pools {[young] [210.6mb]->[220.2mb]/[266.2mb]}{[survivor] [0b]->[0b]/[33.2mb]}{[old] [1.6gb]->[1.6gb]/[1.6gb]}
[2020-06-23T12:09:57,706][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361342] overhead, spent [5.7s] collecting in the last [5.9s]
[2020-06-23T12:10:02,124][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361343] overhead, spent [4.2s] collecting in the last [4.4s]
[2020-06-23T12:10:08,361][INFO ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][old][24361344][358351] duration [5.8s], collections [1]/[6.2s], total [5.8s]/[13.7d], memory [1.8gb]->[1.8gb]/[1.9gb], all_pools {[young] [209.5mb]->[208.4mb]/[266.2mb]}{[survivor] [0b]->[0b]/[33.2mb]}{[old] [1.6gb]->[1.6gb]/[1.6gb]}
[2020-06-23T12:10:08,361][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361344] overhead, spent [5.8s] collecting in the last [6.2s]
[2020-06-23T12:10:13,650][WARN ][o.e.m.j.JvmGcMonitorService] [183.129.160.140] [gc][24361346] overhead, spent [4.1s] collecting in the last [4.1s]

确实发现每隔几秒钟就GC一次!

看了默认2G的堆内存需要调高些了。

/etc/elasticsearch/jvm.options

-Xms3048M
-Xmx3048M

鉴于服务器空余内存不多,就先加1GB上去看看效果。

systemctl stop elasticsearch

systemctl start elasticsearch

重启后过3分钟看top -d 1,ES 的CPU占用果然正常多了。

相关文章

网友评论

      本文标题:2020-06-22 记一次ES占用CPU异常高的问题处理

      本文链接:https://www.haomeiwen.com/subject/qphxfktx.html