美文网首页Apache Kafka
Kafka常用JMX监控指标整理【实战笔记】

Kafka常用JMX监控指标整理【实战笔记】

作者: 瓜农老梁 | 来源:发表于2019-12-08 12:36 被阅读0次
    目录
    一、系统相关指标
二、GC相关指标
三、JVM相关指标
四、Topic相关指标
五、Broker相关指标
    一、系统相关指标

    1.系统信息收集
    java.lang:type=OperatingSystem

    {"freePhysicalMemorySize":"806023168","maxFileDescriptorCount":"4096","openFileDescriptorCount":"283","processCpuLoad":"0.0017562901839817224","systemCpuLoad":"0.014336627412954635","systemLoadAverage":"0.37"}

    2.Thread信息收集
    java.lang:type=Threading

    {"peakThreadCount":"88","threadCount":"74"}

    3.获取mmaped和direct空间

    通过BufferPoolMXBean获取used、capacity、count

    二、GC相关指标

    1.Young GC
    java.lang:type=GarbageCollector,name=G1 Young Generation

    {"collectionCount":"534","collectionTime":"8258"}

    2.Old GC
    java.lang:type=GarbageCollector,name=G1 Old Generation

    {"collectionCount":"0","collectionTime":"0"}
    三、JVM相关指标

    通过MemoryMXBean获取JVM相关信息HeapMemoryUsage和NonHeapMemoryUsage；通过MemoryPoolMXBean获取其他JVM内存空间指标，例如：Metaspace、Codespace等

    四、Topic相关指标

    1.Topic消息入站速率（Byte）
    kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec,topic=" + topic

    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}

    2.Topic消息出站速率（Byte）
    kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec,topic=" + topic

    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}

    3.Topic请求被拒速率
    kafka.server:type=BrokerTopicMetrics,name=BytesRejectedPerSec,topic=" + topic

    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}
    4.Topic失败拉去请求速率

    kafka.server:type=BrokerTopicMetrics,name=FailedFetchRequestsPerSec,topic=" + topic;

    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}
    5.Topic发送请求失败速率

    kafka.server:type=BrokerTopicMetrics,name=FailedProduceRequestsPerSec,topic=" + topic

    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}
    6.Topic消息入站速率（message）

    kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec,topic=" + topic
    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}

    五、Broker相关指标
    1.Log flush rate and time

    kafka.log:type=LogFlushStats,name=LogFlushRateAndTimeMs

    {"50thPercentile":"1.074103","75thPercentile":"1.669793","95thPercentile":"6.846556","98thPercentile":"6.846556","999thPercentile":"6.846556","99thPercentile":"6.846556","count":"19","max":"6.846556","mean":"1.628646052631579","min":"0.512879","stdDev":"1.6007003364105892"}
    2.同步失效的副本数

    kafka.server:type=ReplicaManager,name=UnderReplicatedPartitions

    {"value":"0"}
    3.消息入站速率（消息数）

    kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec

    {"count":"86845","fifteenMinuteRate":"0.6456600497006455","fiveMinuteRate":"0.6444164288097876","meanRate":"0.5314899330400695","oneMinuteRate":"0.6494649408329609"}
    4.消息入站速率（Byte）

    kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec

    {"count":"57302357","fifteenMinuteRate":"379.11342092748146","fiveMinuteRate":"371.8482236385939","meanRate":"351.37122686037435","oneMinuteRate":"351.8348952308101"}
    5.消息出站速率（Byte）

    kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec

    {"count":"246","fifteenMinuteRate":"4.508738367219028E-34","fiveMinuteRate":"1.4721921790135324E-98","meanRate":"0.0015031168286836175","oneMinuteRate":"2.964393875E-314"}
    6.请求被拒速率

    kafka.server:type=BrokerTopicMetrics,name=BytesRejectedPerSec

    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}
    7.失败拉去请求速率

    kafka.server:type=BrokerTopicMetrics,name=FailedFetchRequestsPerSec

    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}
    8.发送请求失败速率

    kafka.server:type=BrokerTopicMetrics,name=FailedProduceRequestsPerSec

    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}
    9.Leader副本数

    kafka.server:type=ReplicaManager,name=LeaderCount

    {"value":"92"}
    10.Partition数量

    kafka.server:type=ReplicaManager,name=PartitionCount

    {"value":"135"}
    11.下线Partition数量

    kafka.controller:type=KafkaController,name=OfflinePartitionsCount

    {"value":"0"}
    12.Broker网络处理线程空闲率

    kafka.server:type=KafkaRequestHandlerPool,name=RequestHandlerAvgIdlePercent

    {"count":"164506926671008","fifteenMinuteRate":"0.9999327359820058","fiveMinuteRate":"1.0000290054537715","meanRate":"0.9998854371393514","oneMinuteRate":"1.0007836499581673"}
    13.Leader选举比率

    kafka.controller:type=ControllerStats,name=LeaderElectionRateAndTimeMs

    {"count":"7","fifteenMinuteRate":"5.134993718576819E-82","fiveMinuteRate":"6.882658450509451E-240","meanRate":"4.2525243043608314E-5","oneMinuteRate":"2.964393875E-314"}
    14.Unclean Leader选举比率

    kafka.controller:type=ControllerStats,name=UncleanLeaderElectionsPerSec

    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}
    15.Controller存活

    kafka.controller:type=KafkaController,name=ActiveControllerCount

    {"value":"1"}
    16.请求速率

    kafka.network:type=RequestMetrics,name=RequestsPerSec,request=Produce

    {"count":"83233","fifteenMinuteRate":"0.6303485369828705","fiveMinuteRate":"0.6357199085092445","meanRate":"0.5046486472186744","oneMinuteRate":"0.6563203475530601"}
    17.Consumer拉取速率

    kafka.network:type=RequestMetrics,name=RequestsPerSec,request=FetchConsumer

    {"count":"125796","fifteenMinuteRate":"1.14193044007404E-33","fiveMinuteRate":"7.699516480260211E-100","meanRate":"0.7623419964866819","oneMinuteRate":"2.964393875E-314"}
    18.Follower拉去速率

    kafka.network:type=RequestMetrics,name=RequestsPerSec,request=FetchFollower

    {"count":"375108","fifteenMinuteRate":"2.302746562040189","fiveMinuteRate":"2.292459728166488","meanRate":"2.2721808581484693","oneMinuteRate":"2.2814260196672973"}
    19.Request total time

    kafka.network:type=RequestMetrics,name=TotalTimeMs,request=Produce

    {"50thPercentile":"1.0","75thPercentile":"1.0","95thPercentile":"2.0","98thPercentile":"2.0","999thPercentile":"28.0","99thPercentile":"4.0","count":"83384","max":"48.0","mean":"1.2344934279957787","min":"0.0","stdDev":"1.1783192073287214"}
    20.Consumer fetch total time

    kafka.network:type=RequestMetrics,name=TotalTimeMs,request=FetchConsumer

    {"50thPercentile":"500.0","75thPercentile":"501.0","95thPercentile":"501.0","98thPercentile":"501.0","999thPercentile":"501.971","99thPercentile":"501.0","count":"125796","max":"535.0","mean":"499.83123469744663","min":"0.0","stdDev":"17.138716708632025"}
    21.Follower fetch total time

    kafka.network:type=RequestMetrics,name=TotalTimeMs,request=FetchFollower

    {"50thPercentile":"500.0","75thPercentile":"500.0","95thPercentile":"501.0","98thPercentile":"501.0","999thPercentile":"507.826","99thPercentile":"501.0","count":"375564","max":"532.0","mean":"437.79763502359117","min":"0.0","stdDev":"148.25999023472986"}
    22.Time the follower fetch request waits in the request queue

    kafka.network:type=RequestMetrics,name=RequestQueueTimeMs,request=FetchFollower

    {"50thPercentile":"0.0","75thPercentile":"0.0","95thPercentile":"0.0","98thPercentile":"0.0","999thPercentile":"0.0","99thPercentile":"0.0","count":"376206","max":"28.0","mean":"0.0010260336092459982","min":"0.0","stdDev":"0.1282889653905258"}
    23.Time the Consumer fetch request waits in the request queue

    kafka.network:type=RequestMetrics,name=RequestQueueTimeMs,request=FetchConsumer

    {"50thPercentile":"0.0","75thPercentile":"0.0","95thPercentile":"0.0","98thPercentile":"0.0","999thPercentile":"0.0","99thPercentile":"0.0","count":"125796","max":"24.0","mean":"0.0018124582657636174","min":"0.0","stdDev":"0.18122860552537737"}
    24.Time the Produce fetch request waits in the request queue

    kafka.network:type=RequestMetrics,name=RequestQueueTimeMs,request=Produce

    {"50thPercentile":"0.0","75thPercentile":"0.0","95thPercentile":"0.0","98thPercentile":"0.0","999thPercentile":"0.0","99thPercentile":"0.0","count":"83704","max":"12.0","mean":"2.6283092803211315E-4","min":"0.0","stdDev":"0.042892540270754634"}
    25.Broker I/O工作处理线程空闲率

    kafka.network:type=SocketServer,name=NetworkProcessorAvgIdlePercent

    {"value":"1.0015540075894207"}
    26.ISR变化速率

    kafka.server:type=ReplicaManager,name=IsrShrinksPerSec

    {"count":"0","fifteenMinuteRate":"0.0","fiveMinuteRate":"0.0","meanRate":"0.0","oneMinuteRate":"0.0"}

    作者老梁，哈啰出行高级技术专家，参与了《RocketMQ技术内幕》审稿工作。专注后端中间件方向，已陆续发表RocketMQ系列、Kafka系列、gRPC系列、Sentinel系列、Java NIO系列。其中RocketMQ系列已发表40余篇。源码、实战、原理、调优期待与你一起学习。


    公众号.jpg
    个人微信.png

    相关文章

      网友评论

        本文标题：Kafka常用JMX监控指标整理【实战笔记】

        本文链接：https://www.haomeiwen.com/subject/wblmgctx.html

        延伸阅读

        深度阅读

        • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

        栏目导航

        热点阅读

        Apache Kafka

        关于我们|服务条款|联系我们|Kafka常用JMX监控指标整理【实战笔记】|投稿指南|网站地图|RSS订阅|排版工具|手机版

        提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

        Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

        备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

        本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

        所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！