RocketMQ 生产者发送消息失败分析 [TIMEOUT_CL

作者: awker | 来源:发表于2019-12-02 21:15 被阅读0次

RocketMQ 生产者发送消息失败分析 [TIMEOUT_CL
RocketMQ生产者
RocketMQ 基本概念
2. RocketMQ进阶1_架构与集群部署模式
RocketMQ（二）RocketMQ高级特性
消息队列之RabbitMQ-消息确认机制
2020-03-12
RocketMQ消息丢失解决方案：同步刷盘+手动提交
RocketMQ消息发送源码分析
RocketMq 实际案例--普通消息的发送

问题：
org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2 DESC: [TIMEOUT_CLEAN_QUEUE]broker busy, start flow control for a while, period in queue: 201ms, size of queue: 0 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/

排查：
服务端 broker-a master 日志 store.log：

2019-12-02 09:39:46 WARN ReadSocketService - ha housekeeping, found this connection[/10.2.16.49:35392] expired, 20013
2019-12-02 09:39:46 INFO ReadSocketService - ReadSocketService service end
2019-12-02 09:39:46 INFO AcceptSocketService - HAService receive new connection, /10.2.16.49:35504
2019-12-02 09:39:46 INFO ReadSocketService - ReadSocketService service started
2019-12-02 09:39:46 INFO WriteSocketService - WriteSocketService service started
2019-12-02 09:39:46 INFO WriteSocketService - WriteSocketService service end

服务端 broker-a slave 日志 store.log：

2019-12-02 09:39:46 INFO HAClient - HAClient, processReadEvent read socket < 0
2019-12-02 09:39:46 WARN HAClient - HAClient, housekeeping, found this connection[10.2.16.48:10912] expired, 1575250786458
2019-12-02 09:39:46 WARN HAClient - HAClient, master not response some time, so close connection

生产者发送消息 --> broker-a master（同步到 broker-a slave 后，返回 ack 给生产者）--> 消费者取到消息
生产者发送消息 --> broker-a master（同步到 broker-a slave 时失败，没有返回 ack 给生产者）--> 消费者没有取到消息

解决：

修改同步方式为 ASYNC_MASTER，存在丢消息风险
修改 slave 发送心跳时间

这里采用 修改 slave 发送心跳时间 的方法

// 查看 rocketmq 集群信息
# ./bin/mqadmin clusterList
#Cluster Name     #Broker Name            #BID  #Addr                  #Version                #InTPS(LOAD)       #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
DefaultCluster    broker-a                0     10.2.16.48:10911       V4_4_0                   0.00(0,0ms)         0.00(0,0ms)          0 2082.56 0.1278
DefaultCluster    broker-a                1     10.2.16.49:10911       V4_4_0                   0.00(0,0ms)         0.00(0,0ms)          0 2082.56 0.1408
DefaultCluster    broker-b                0     10.2.16.50:10911       V4_4_0                   0.00(0,0ms)         0.00(0,0ms)          0 2082.56 0.2458
DefaultCluster    broker-b                1     10.2.16.51:10911       V4_4_0                   0.00(0,0ms)         0.00(0,0ms)          0 2082.56 0.2515

// 修改 broker-a  slave 发送心跳时间
# ./bin/mqadmin updateBrokerConfig -b 10.2.16.49:10911 -k haSendHeartbeatInterval -v 4000
update broker config success, 10.2.16.49:10911

// 修改 broker-b slave 发送心跳时间
# ./bin/mqadmin updateBrokerConfig -b 10.2.16.51:10911 -k haSendHeartbeatInterval -v 4000
update broker config success, 10.2.16.51:10911

// 查看 broker-a slave 发送心跳时间是否修改成功
# ./bin/mqadmin getBrokerConfig -b 10.2.16.49:10911 | grep haSend
haSendHeartbeatInterval                           =  4000

// 查看 broker-b slave 发送心跳时间是否修改成功
# ./bin/mqadmin getBrokerConfig -b 10.2.16.51:10911 | grep haSend
haSendHeartbeatInterval                           =  4000

参考链接