Netty的内存泄漏问题

作者: smlrole | 来源:发表于2019-10-09 15:41 被阅读0次

Netty的内存泄漏问题
每周阅读（7/25）
Netty内存泄漏问题排查
关于Netty的ByteBuff内存泄漏问题
Xcode调试工具
Netty堆外内存泄漏排查，这一篇全讲清楚了
JVM
三个方法帮助解决Android内存泄漏问题
netty 解码小坑
安卓内存泄漏

在测试netty时发现这个问题

LEAK: ByteBuf.release() was not called before it's garbage-collected. Enable advanced leak reporting to find out where the leak occurred. To enable advanced leak reporting, specify the JVM option '-Dio.netty.leakDetectionLevel=advanced' or call ResourceLeakDetector.setLevel() See http://netty.io/wiki/reference-counted-objects.html for more information. |

两种方式改变日志级别

ResourceLeakDetector.setLevel(ResourceLeakDetector.Level.ADVANCED);
-Dio.netty.leakDetectionLevel=advanced

继续测试：

 LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 5
#5:
    io.netty.buffer.AdvancedLeakAwareByteBuf.toString(AdvancedLeakAwareByteBuf.java:697)
    io.netty.handler.codec.http.websocketx.TextWebSocketFrame.text(TextWebSocketFrame.java:94)
    com.qingqing.livebroker.server.handler.WebsocketMsgHandler.channelRead(WebsocketMsgHandler.java:76)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.handler.codec.http.websocketx.WebSocketServerProtocolHandler$1.channelRead(WebSocketServerProtocolHandler.java:146)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:244)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
    io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
    io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
    io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
    io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
    io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
    io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
    io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    java.lang.Thread.run(Thread.java:745)
#4:
    io.netty.buffer.AdvancedLeakAwareByteBuf.release(AdvancedLeakAwareByteBuf.java:45)
    io.netty.buffer.DefaultByteBufHolder.release(DefaultByteBufHolder.java:73)
    io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:59)
    io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:91)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.handler.codec.http.websocketx.WebSocketServerProtocolHandler$1.channelRead(WebSocketServerProtocolHandler.java:146)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:244)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
    io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
    io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
    io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
    io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
    io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
    io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
    io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    java.lang.Thread.run(Thread.java:745)
#3:
    io.netty.buffer.AdvancedLeakAwareByteBuf.retain(AdvancedLeakAwareByteBuf.java:709)
    io.netty.buffer.DefaultByteBufHolder.retain(DefaultByteBufHolder.java:61)
    io.netty.handler.codec.http.websocketx.WebSocketFrame.retain(WebSocketFrame.java:76)
    io.netty.handler.codec.http.websocketx.TextWebSocketFrame.retain(TextWebSocketFrame.java:109)
    io.netty.handler.codec.http.websocketx.TextWebSocketFrame.retain(TextWebSocketFrame.java:25)
    io.netty.handler.codec.http.websocketx.WebSocketProtocolHandler.decode(WebSocketProtocolHandler.java:37)
    io.netty.handler.codec.http.websocketx.WebSocketServerProtocolHandler.decode(WebSocketServerProtocolHandler.java:114)
    io.netty.handler.codec.http.websocketx.WebSocketServerProtocolHandler.decode(WebSocketServerProtocolHandler.java:51)
    io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.handler.codec.http.websocketx.WebSocketServerProtocolHandler$1.channelRead(WebSocketServerProtocolHandler.java:146)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:244)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
    io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
    io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
    io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
    io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
    io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
    io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
    io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    java.lang.Thread.run(Thread.java:745)
#2:
    io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:625)
    io.netty.handler.codec.http.websocketx.Utf8Validator.check(Utf8Validator.java:74)
    io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.checkUTF8String(WebSocket08FrameDecoder.java:464)
    io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.decode(WebSocket08FrameDecoder.java:337)
    io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:370)
    io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:230)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
    io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
    io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
    io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
    io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
    io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
    io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
    io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    java.lang.Thread.run(Thread.java:745)
#1:
    io.netty.buffer.AdvancedLeakAwareByteBuf.setByte(AdvancedLeakAwareByteBuf.java:253)
    io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.unmask(WebSocket08FrameDecoder.java:429)
    io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.decode(WebSocket08FrameDecoder.java:302)
    io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:370)
    io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:230)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
    io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
    io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
    io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
    io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
    io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
    io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
    io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    java.lang.Thread.run(Thread.java:745)
Created at:
    io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:55)
    io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
    io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
    io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:83)
    io.netty.handler.codec.http.websocketx.WebSocket08FrameDecoder.decode(WebSocket08FrameDecoder.java:263)
    io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:370)
    io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:230)
    io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
    io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
    io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
    io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
    io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
    io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
    io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
    io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
    io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
    io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    java.lang.Thread.run(Thread.java:745) |

ByteBuf 是Netty中主要用来数据byte[]的封装类，主要分为Heap ByteBuf 和 Direct ByteBuf。
为了减少内存的分配回收以及产生的内存碎片，Netty提供了PooledByteBufAllocator 用来分配可回收的ByteBuf，可以把PooledByteBufAllocator看做一个池子，需要的时候从里面获取ByteBuf，用完了放回去，以此提高性能。当然与之对应的还有 UnpooledByteBufAllocator，顾名思义Unpooled就是不会放到池子里，所以根据该分配器分配的ByteBuf，不需要放回池子有JVM自己GC回收。
在netty中，根据ChannelHandlerContext 和 Channel获取的Allocator默认都是Pooled，所以需要再合适的时机对其进行释放，避免造成内存泄漏。
————————————————
原文链接：https://blog.csdn.net/u012807459/article/details/77259869

错误的第三行可以定位到是我的 ChannelInboundHandlerAdapter 没有对获取到 byte进行释放

// 第一种
textWebSocketFrame.release();
// 第二种
//ctx.fireChannelRead(msg);

以上两种方式都可以释放ByteBuf。

Netty的内存泄漏问题
在测试netty时发现这个问题两种方式改变日志级别 ResourceLeakDetector.setLevel(...
每周阅读（7/25）
追踪 Netty 异常占用堆外内存的经验分享LeanCloud团队关于Netty堆外内存泄漏的调查 MongoDB...
Netty内存泄漏问题排查
1. 问题描述手机收到告警 “主机内存不足, 使用率高达 90.31%, 使用了 3.49G”。线上机器查看日...
关于Netty的ByteBuff内存泄漏问题
之前做的东华车管数据采集平台总是发生数据丢失的情况，虽然不频繁但是还是要关注一下原因，于是今天提高了Netty的L...
Xcode调试工具
一.静态内存分析工具编译阶段查找内存泄漏等问题 1.常见内存泄漏问题常见的内存泄漏除了循环引用，CoreFou...
Netty堆外内存泄漏排查，这一篇全讲清楚了
上篇文章介绍了Netty内存模型原理，由于Netty在使用不当会导致堆外内存泄漏，网上关于这方面的资料比较少，所以...
JVM
直接内存使用场景：Unsafe类、NIO零拷贝、Netty的零拷贝、JNI 优点：性能更高缺点：内存泄漏难排查...
三个方法帮助解决Android内存泄漏问题
三个方法帮助解决Android内存泄漏问题最近自己遇到了好几个内存泄漏的问题，也帮同事解决了几个内存泄漏的问题记...
netty 解码小坑
1. in.readBytes 导致堆外内存泄漏使用netty 中偶现 LEAK: ByteBuf.releas...
安卓内存泄漏
Android 内存泄漏总结内存管理的目的就是让我们在开发中怎么有效的避免我们的应用出现内存泄漏的问题。内存泄漏...