概述
这里介绍redis cluster模式下master水平扩容支撑海量数据。
master水平扩容
redis cluster通过master的水平扩容来横向扩展读写吞吐量，从而支撑更多的海量数据。
并发量统计
一般情况下，
- 单机redis的读吞吐是5w/s，写吞吐2w/s
- 扩展5台master redis节点，支撑读吞吐量25/s QPS，写吞吐10w/s QPS
redis扩容
- 加入新的master节点
mkdir -p /var/redis/7007
port 7007
cluster-enabled yes
cluster-config-file /etc/redis-cluster/node-7007.conf
cluster-node-timeout 15000
daemonize yes
pidfile /var/run/redis_7007.pid
dir /var/redis/7007
logfile /var/log/redis/7007.log
bind 192.168.201.35
appendonly yes
老规矩，加一个7007.conf和redis_7007启动脚本
在7007端口上启动新的redis实例：
[root@dlsc-201-33 ~]# redis-trib.rb add-node 192.168.201.35:7007 192.168.201.33:7001
>>> Adding node 192.168.201.35:7007 to cluster 192.168.201.33:7001
>>> Performing Cluster Check (using node 192.168.201.33:7001)
S: 1617e0e394b5c74be6a6b804980c2c857d352522 192.168.201.33:7001
slots: (0 slots) slave
replicates 7142f5825e214522d12a3d01b50c79e620a00a2d
M: 7142f5825e214522d12a3d01b50c79e620a00a2d 192.168.201.34:7004
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: c8d5cccb9263263512cb26e3fc776173899a99cf 192.168.201.35:7006
slots: (0 slots) slave
replicates 0e52767dbadee66cf788494501fdd7f23c66b935
S: 46967bbb36422de8af09545dd670a07ccb0cef52 192.168.201.33:7002
slots: (0 slots) slave
replicates d70eae6f6ae651b8af3f07a3137716f7dfaf2d84
M: d70eae6f6ae651b8af3f07a3137716f7dfaf2d84 192.168.201.35:7005
slots:10923-16383 (5461 slots) master
1 additional replica(s)
M: 0e52767dbadee66cf788494501fdd7f23c66b935 192.168.201.34:7003
slots:5461-10922 (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.201.35:7007 to make it join the cluster.
[OK] New node added correctly.
[root@dlsc-201-33 ~]# redis-trib.rb check 192.168.201.33:7001
>>> Performing Cluster Check (using node 192.168.201.33:7001)
S: 1617e0e394b5c74be6a6b804980c2c857d352522 192.168.201.33:7001
slots: (0 slots) slave
replicates 7142f5825e214522d12a3d01b50c79e620a00a2d
M: 7142f5825e214522d12a3d01b50c79e620a00a2d 192.168.201.34:7004
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: c8d5cccb9263263512cb26e3fc776173899a99cf 192.168.201.35:7006
slots: (0 slots) slave
replicates 0e52767dbadee66cf788494501fdd7f23c66b935
M: a5979b12a7a9a50581be0d7f64cd97e9e0e79dd9 192.168.201.35:7007
slots: (0 slots) master
0 additional replica(s)
S: 46967bbb36422de8af09545dd670a07ccb0cef52 192.168.201.33:7002
slots: (0 slots) slave
replicates d70eae6f6ae651b8af3f07a3137716f7dfaf2d84
M: d70eae6f6ae651b8af3f07a3137716f7dfaf2d84 192.168.201.35:7005
slots:10923-16383 (5461 slots) master
1 additional replica(s)
M: 0e52767dbadee66cf788494501fdd7f23c66b935 192.168.201.34:7003
slots:5461-10922 (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
连接到新的redis实例上，cluster nodes作为了一个新的master，确认已经加入了集群。但是没有hash slot，需要通过reshard把数据迁移过去。
2.reshard数据
resharding会把一部分hash slot从一些node上迁移到另外一些node上
redis-trib.rb reshard 192.168.201.33:7001
要把之前3个master上，总共4096个hashslot迁移到新的第四个master上去
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? a5979b12a7a9a50581be0d7f64cd97e9e0e79dd9
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:7142f5825e214522d12a3d01b50c79e620a00a2d
Source node #2:d70eae6f6ae651b8af3f07a3137716f7dfaf2d84
Source node #3:0e52767dbadee66cf788494501fdd7f23c66b935
Source node #4:done
Do you want to proceed with the proposed reshard plan (yes/no)?yes
- 添加node作为slave
192.168.201.35
mkdir -p /var/redis/7008
port 7008
cluster-enabled yes
cluster-config-file /etc/redis-cluster/node-7008.conf
cluster-node-timeout 15000
daemonize yes
pidfile /var/run/redis_7008.pid
dir /var/redis/7008
logfile /var/log/redis/7008.log
bind 192.168.201.35
appendonly yes
redis-trib.rb add-node --slave --master-id 7142f5825e214522d12a3d01b50c79e620a00a2d 192.168.201.35:7008 192.168.201.33:7001
- 删除node
先用resharding将数据都迁移到其他节点，使得node为空之后，才能执行删除操作
[root@dlsc-201-33 ~]# redis-trib.rb reshard 192.168.201.33:7001
>>> Performing Cluster Check (using node 192.168.201.33:7001)
S: 1617e0e394b5c74be6a6b804980c2c857d352522 192.168.201.33:7001
slots: (0 slots) slave
replicates 7142f5825e214522d12a3d01b50c79e620a00a2d
M: 7142f5825e214522d12a3d01b50c79e620a00a2d 192.168.201.34:7004
slots:2731-5460 (2730 slots) master
1 additional replica(s)
S: c8d5cccb9263263512cb26e3fc776173899a99cf 192.168.201.35:7006
slots: (0 slots) slave
replicates 0e52767dbadee66cf788494501fdd7f23c66b935
M: a5979b12a7a9a50581be0d7f64cd97e9e0e79dd9 192.168.201.35:7007
slots:0-2730,5461-8191,10923-13652 (8192 slots) master
0 additional replica(s)
S: 46967bbb36422de8af09545dd670a07ccb0cef52 192.168.201.33:7002
slots: (0 slots) slave
replicates d70eae6f6ae651b8af3f07a3137716f7dfaf2d84
M: d70eae6f6ae651b8af3f07a3137716f7dfaf2d84 192.168.201.35:7005
slots:13653-16383 (2731 slots) master
1 additional replica(s)
M: 0e52767dbadee66cf788494501fdd7f23c66b935 192.168.201.34:7003
slots:8192-10922 (2731 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 8192
What is the receiving node ID? 0e52767dbadee66cf788494501fdd7f23c66b935
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:a5979b12a7a9a50581be0d7f64cd97e9e0e79dd9
Source node #2:done
redis-trib.rb del-node 192.168.201.35:7007 a5979b12a7a9a50581be0d7f64cd97e9e0e79dd9
>>> Removing node a5979b12a7a9a50581be0d7f64cd97e9e0e79dd9 from cluster 192.168.201.35:7007
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
注意：2个是1365，1个是1366
如果清空的是master节点的hashslot时，redis cluster就会自动将其slave挂载到其他master节点上去。
网友评论