美文网首页
linux下RocketMQ的集群搭建

linux下RocketMQ的集群搭建

作者: 牙齿不帅 | 来源:发表于2020-07-23 09:29 被阅读0次

    部署的模式为：2m-2s-async　两主两从－异步同步
    节点1：10.10.94.43【a master】【 b slave】
    节点2：10.10.94.44【b master】【 a slave】

    1.下载mq的bin包，并上传到两个不同的linux服务器上。

    下载链接：http://rocketmq.apache.org/release_notes/release-notes-4.3.2/

    2.解压后移动到/usr/local/rocketmq目录下。
    mv rocketmq-all-4.3.2-bin-release /usr/local/rocketmq
    3.创建需要的目录。

    节点1【master】执行：

     mkdir -p /usr/local/rocketmq/store/broker-a /usr/local/rocketmq/store/broker-a/nsumequeue /usr/local/rocketmq/store/broker-a/commitlog /usr/local/rocketmq/store/broker-a/index /usr/local/rocketmq/logs /usr/local/rocketmq/store/broker-b-s /usr/local/rocketmq/store/broker-b-s/nsumequeue /usr/local/rocketmq/store/broker-b-s/commitlog /usr/local/rocketmq/store/broker-b-s/index

    节点2【slave】执行：

    mkdir -p /usr/local/rocketmq/store/broker-a-s /usr/local/rocketmq/store/broker-a-s/nsumequeue /usr/local/rocketmq/store/broker-a-s/commitlog /usr/local/rocketmq/store/broker-a-s/index /usr/local/rocketmq/logs /usr/local/rocketmq/store/broker-b /usr/local/rocketmq/store/broker-b/nsumequeue /usr/local/rocketmq/store/broker-b/commitlog /usr/local/rocketmq/store/broker-b/index
    4.启动nameserver服务器。

    nameserver没有需要修改的配置文件，默认端口为9876，且nameserver服务器互相之间是不通信的。节点1，2分别执行如下命令：

    nohup sh /usr/local/rocketmq/bin/mqnamesrv >/usr/local/rocketmq/logs/mqnamesrv.log 2>&1 &

    jps显示NamesrvStartup已启动如下：

    # jps -l
15707 org.apache.rocketmq.namesrv.NamesrvStartup
29499 sun.tools.jps.Jps
    5.修改两个节点的配置文件，位于/usr/local/rocketmq/conf/2m-2s-async目录，目的是为了启动broker服务。

    节点1【a master】【b slave】如下：

    vim broker-a.properties

    #所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字，名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerName=broker-a
#0 表示 Master，>0 表示 Slave
brokerId=0
#nameServer地址，分号分割
namesrvAddr=10.10.94.43:9876;10.10.94.44:9876
brokerIP1=10.10.94.43
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10911
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store/broker-a
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store/broker-a/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store/broker-a/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store/broker-a/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

    vim broker-b-s.properties

    #所属集群名字
brokerClusterName=rocketmq-cluster
brokerName=broker-b
#broker名字，名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-b,他的slave也叫broker-b
#0 表示 Master，>0 表示 Slave
brokerId=1
#nameServer地址，分号分割
namesrvAddr=10.10.94.43:9876;10.10.94.44:9876
brokerIP1=10.10.94.43
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10920
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store/broker-b-s
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store/broker-b-s/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store/broker-b-s/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store/broker-b-s/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128

    节点2【b master】【a slave】

    vim broker-b.properties #基本和节点1的相同除了ip和brokerId

    #所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字，名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerName=broker-b
#0 表示 Master，>0 表示 Slave
brokerId=0
#nameServer地址，分号分割
namesrvAddr=10.10.94.43:9876;10.10.94.44:9876
brokerIP1=10.10.94.44
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10911
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store/broker-b
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store/broker-b/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store/broker-b/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store/broker-b/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
vim broker-a-s.properties #内容和节点1的broker-b-s.properties配置相同，此处忽略

    vim broker-a-s.properties #基本和节点1的相同除了ip和brokerId

    brokerClusterName=rocketmq-cluster
brokerName=broker-a
#broker名字，名字可重复,为了管理,每个master起一个名字,他的slave同他,eg:Amaster叫broker-a,他的slave也叫broker-a
brokerId=1
#nameServer地址，分号分割
namesrvAddr=10.10.94.43:9876;10.10.94.44:9876
brokeIP1=10.10.94.44
#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口,
listenPort=10920
#删除文件时间点，默认凌晨 4点
deleteWhen=04
#文件保留时间，默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条，根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store/broker-a-s
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store/broker-a-s/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store/broker-a-s/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store/broker-a-s/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SLAVE
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128 under the License.
    6.启动broker服务。

    节点1，启动broker-a：

    nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a.properties > /usr/local/rocketmq/logs/broker-a.log 2>&1 &

    ps查看启动成功

    # ps -ef|grep broker
root     31704 22886  0 15:23 pts/1    00:00:00 sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a.properties
root     31708 31704  0 15:23 pts/1    00:00:00 sh /usr/local/rocketmq/bin/runbroker.sh org.apache.rocketmq.broker.BrokerStartup -c /usr/local/rocketmq/conf/2m-2s-async/broker-a.properties
root     31711 31708 47 15:23 pts/1    00:00:05 /usr/java/jdk1.8.0_131/bin/java -server -Xms8g -Xmx8g -Xmn4g -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=15g -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/usr/java/jdk1.8.0_131/jre/lib/ext:/usr/local/rocketmq/bin/../lib -cp .:/usr/local/rocketmq/bin/../conf:.:/usr/java/jdk1.8.0_131/jre/lib/rt.jar:/usr/java/jdk1.8.0_131/lib/tools.jar org.apache.rocketmq.broker.BrokerStartup -c /usr/local/rocketmq/conf/2m-2s-async/broker-a.properties
root     31865 22886  0 15:23 pts/1    00:00:00 grep --color=auto broker

    节点1，启动broker-b-s：

    nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-b-s.properties > /usr/local/rocketmq/logs/broker-b-s.log 2>&1 &

    # ps -ef|grep mq
root     27614 10167  0 14:40 pts/0    00:00:00 sh /usr/local/rocketmq/bin/mqnamesrv
root     27618 27614  0 14:40 pts/0    00:00:00 sh /usr/local/rocketmq/bin/runserver.sh org.apache.rocketmq.namesrv.NamesrvStartup
root     27620 27618  0 14:40 pts/0    00:00:10 /home/jdk1.8.0_131/bin/java -server -Xms4g -Xmx4g -Xmn2g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8 -XX:-UseParNewGC -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX:+PrintGCDetails -XX:-OmitStackTraceInFastThrow -XX:-UseLargePages -Djava.ext.dirs=/home/jdk1.8.0_131/jre/lib/ext:/usr/local/rocketmq/bin/../lib -cp .:/usr/local/rocketmq/bin/../conf::/home/jdk1.8.0_131/lib/dt.jar:/home/jdk1.8.0_131/lib/tools.jar:/home/jdk1.8.0_131/jre/lib/dt.jar org.apache.rocketmq.namesrv.NamesrvStartup
root     28454 10167  0 16:07 pts/0    00:00:00 sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a-s.properties
root     28458 28454  0 16:07 pts/0    00:00:00 sh /usr/local/rocketmq/bin/runbroker.sh org.apache.rocketmq.broker.BrokerStartup -c /usr/local/rocketmq/conf/2m-2s-async/broker-a-s.properties
root     28461 28458 82 16:07 pts/0    00:00:05 /home/jdk1.8.0_131/bin/java -server -Xms8g -Xmx8g -Xmn4g -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8 -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m -XX:-OmitStackTraceInFastThrow -XX:+AlwaysPreTouch -XX:MaxDirectMemorySize=15g -XX:-UseLargePages -XX:-UseBiasedLocking -Djava.ext.dirs=/home/jdk1.8.0_131/jre/lib/ext:/usr/local/rocketmq/bin/../lib -cp .:/usr/local/rocketmq/bin/../conf::/home/jdk1.8.0_131/lib/dt.jar:/home/jdk1.8.0_131/lib/tools.jar:/home/jdk1.8.0_131/jre/lib/dt.jar org.apache.rocketmq.broker.BrokerStartup -c /usr/local/rocketmq/conf/2m-2s-async/broker-a-s.properties
root     28536 10167  0 16:07 pts/0    00:00:00 grep --color=auto mq

    节点2，启动broker-b：

    nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-b.properties > /usr/local/rocketmq/logs/broker-b.log 2>&1 &

    节点2，启动broker-a-s：

    nohup sh /usr/local/rocketmq/bin/mqbroker -c /usr/local/rocketmq/conf/2m-2s-async/broker-a-s.properties > /usr/local/rocketmq/logs/broker-a-s.log 2>&1 &
    7.查看集群列表。
    sh /usr/local/rocketmq/bin/mqadmin clusterList -n 10.10.94.44:9876

    Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
#Cluster Name     #Broker Name            #BID  #Addr                  #Version                #InTPS(LOAD)       #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
rocketmq-cluster  broker-a                0     10.10.94.43:10911      V4_3_2                   0.00(0,0ms)         0.00(0,0ms)          0 443185.33 0.6434
rocketmq-cluster  broker-a                1     10.10.94.44:10920      V4_3_2                   0.00(0,0ms)         0.00(0,0ms)          0 443185.33 0.6869
rocketmq-cluster  broker-b                0     10.10.94.44:10911      V4_3_2                   0.00(0,0ms)         0.00(0,0ms)          0 443185.33 0.6869
rocketmq-cluster  broker-b                1     10.10.94.43:10920      V4_3_2                   0.00(0,0ms)         0.00(0,0ms)          0 443185.33 0.6434
    最后：

    启动时可能会出现内存不足的错误，如下，因为mq默认分配的很大：

    Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000005c0000000, 8589934592, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 8589934592 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /usr/local/rocketmq/conf/2m-2s-async/hs_err_pid28713.log

    调小内存，即可解决：

    如果单台启动了多个broker，先关闭
# ./bin/mqshutdown broker
# vim bin/runbroker.sh
JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn256m"

    总结：网上的文章很多，但还是自己亲手搭建一下，才能发现问题，更深刻的理解，最好是先了解了ＲＭＱ的架构，才能轻车熟路。

    参考：https://blog.csdn.net/weixin_40533111/article/details/84451219

    相关文章

      网友评论

          本文标题：linux下RocketMQ的集群搭建

          本文链接：https://www.haomeiwen.com/subject/wjszkktx.html

          延伸阅读

          深度阅读

          • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

          栏目导航

          热点阅读

          关于我们|服务条款|联系我们|linux下RocketMQ的集群搭建|投稿指南|网站地图|RSS订阅|排版工具|手机版

          提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

          Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

          备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

          本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

          所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！