Redis集群

作者: Evan_Vivian | 来源:发表于2018-09-29 16:26 被阅读0次

Redis之集群环境搭建
单机(win10)使用Docker搭建Redis集群（Redis
NoSQL Redis 集群架构（Cluster）--Day2
Redis 集群规范
centos防止redis crash的配置
redis集群练习1(redis cluster tutoria
redis集群练习2
redis集群练习3
redis源码阅读
2019-07-04 NoSQL-Day02-Redis Clu

Redis在 3.0 版本以后推出了集群模式。Redis集群是一个可以在多个Redis节点之间进行数据共享的程序集。

安装Redis

yum -y install gcc gcc-c++
cd /opt
wget http://download.redis.io/releases/redis-4.0.11.tar.gz
tar xf redis-4.0.11.tar.gz
cd redis-4.0.11
make

集群模式redis.conf示例

bind 10.211.55.4
daemonize yes

port 7000
dir /opt/redis_cluster/7000/
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

集群所需Redis多实例部署条件

mkdir -p /opt/redis_cluster/{7000,7001,7002,7003,7004,7005,bin}

cp /opt/redis-4.0.11/src/redis-server /opt/redis_cluster/bin/
cp /opt/redis-4.0.11/src/redis-cli /opt/redis_cluster/bin/
cp /opt/redis-4.0.11/src/redis-trib.rb /opt/redis_cluster/bin/
vim /opt/redis_cluster/7000/redis.conf
vim /opt/redis_cluster/7001/redis.conf
vim /opt/redis_cluster/7002/redis.conf
vim /opt/redis_cluster/7003/redis.conf
vim /opt/redis_cluster/7004/redis.conf
vim /opt/redis_cluster/7005/redis.conf

cd /opt/redis_cluster/7000
../bin/redis-server ./redis.conf
cd ../7001
../bin/redis-server ./redis.conf
cd ../7002
../bin/redis-server ./redis.conf
cd ../7003
../bin/redis-server ./redis.conf
cd ../7004
../bin/redis-server ./redis.conf
cd ../7005
../bin/redis-server ./redis.conf

ps -ef|grep redis-server

# root @ Evan in /opt/redis_cluster
$ tree
.
├── 7000
│   ├── appendonly.aof
│   ├── dump.rdb
│   ├── nodes.conf
│   └── redis.conf
├── 7001
│   ├── appendonly.aof
│   ├── dump.rdb
│   ├── nodes.conf
│   └── redis.conf
├── 7002
│   ├── appendonly.aof
│   ├── dump.rdb
│   ├── nodes.conf
│   └── redis.conf
├── 7003
│   ├── appendonly.aof
│   ├── dump.rdb
│   ├── nodes.conf
│   └── redis.conf
├── 7004
│   ├── appendonly.aof
│   ├── dump.rdb
│   ├── nodes.conf
│   └── redis.conf
├── 7005
│   ├── appendonly.aof
│   ├── dump.rdb
│   ├── nodes.conf
│   └── redis.conf
└── bin
    ├── redis-cli
    ├── redis-trib.rb
    └── redis-server


7 directories, 27 files

redis-trib创建集群

Redis的实例全部运行之后，还需要redis-trib.rb工具来完成集群的创建，redis-trib.rb二进制文件在Redis包主目录下的src目录中，运行该工具依赖Ruby环境和gem，因此需要提前安装。

gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
\curl -sSL https://get.rvm.io | bash -s stable

source /usr/local/rvm/scripts/rvm
rvm install 2.4.0
rvm use 2.4.0

gem install redis
/opt/redis_cluster/bin/redis-trib.rb create --replicas 1 10.211.55.4:7000 10.211.55.4:7001 10.211.55.4:7002 10.211.55.4:7003 10.211.55.4:7004 10.211.55.4:7005
##参数create表示创建一个新的集群，--replicas 1表示为每个master创建一个slave。

客户端验证redis集群

/opt/redis_cluster/redis-cli -c -h 10.211.55.4 -p 7000
# -c 表示登录集群模式

10.211.55.4:7000> get foo
-> Redirected to slot [12182] located at 10.211.55.4:7002
"bar"
10.211.55.4:7002> set Evan LoveVivian3
-> Redirected to slot [3330] located at 10.211.55.4:7000
OK
10.211.55.4:7000> set Evan a
OK
10.211.55.4:7000> set Evan ttttttttts
OK
10.211.55.4:7000> set tttttt xxx10002x
-> Redirected to slot [8575] located at 10.211.55.4:7001
OK
10.211.55.4:7001> set tttttt xxx10002xxxxx
OK
10.211.55.4:7001> set txsssa xxx10002xxxxx
OK

使用cluster info命令查看集群状态。###

10.211.55.4:7001> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:8
cluster_my_epoch:2
cluster_stats_messages_ping_sent:7566
cluster_stats_messages_pong_sent:3685
cluster_stats_messages_fail_sent:15
cluster_stats_messages_auth-ack_sent:2
cluster_stats_messages_sent:11268
cluster_stats_messages_ping_received:3685
cluster_stats_messages_pong_received:3722
cluster_stats_messages_fail_received:10
cluster_stats_messages_auth-req_received:2
cluster_stats_messages_received:7419

使用cluster nodes命令查看节点状态。

10.211.55.4:7001> cluster nodes
6f3d4c358463a0e378a5511d7f473e81897e9910 10.211.55.4:7002@17002 master - 0 1535710970000 3 connected 10923-16383
41e6c6785110b8f9521406e3a698b676a0c68d28 10.211.55.4:7003@17003 slave e0c43b03ce8afab1785dff6c83d85a24654e97d3 0 1535710970717 8 connected
0b02dc79c42d047a94d2d5bd12cdf3448fe72647 10.211.55.4:7001@17001 myself,master - 0 1535710969000 2 connected 5461-10922
e0c43b03ce8afab1785dff6c83d85a24654e97d3 10.211.55.4:7000@17000 master - 0 1535710970516 8 connected 0-5460
461f08effa70e742a6f80a597876c11b9f69ebd7 10.211.55.4:7005@17005 slave 6f3d4c358463a0e378a5511d7f473e81897e9910 0 1535710969510 6 connected
5968ecb68bf7671bee2a8af201f14280b2dd6042 10.211.55.4:7004@17004 slave 0b02dc79c42d047a94d2d5bd12cdf3448fe72647 0 1535710970000 5 connected

kill集群所需节点测试

$ ps -ef|grep redis                         
root      8436     1  0 17:47 ?        00:00:00 ../redis-server 10.211.55.4:7000 [cluster]
root      8462     1  0 17:47 ?        00:00:00 ../redis-server 10.211.55.4:7001 [cluster]
root      8488     1  0 17:47 ?        00:00:00 ../redis-server 10.211.55.4:7002 [cluster]
root      8514     1  0 17:47 ?        00:00:00 ../redis-server 10.211.55.4:7003 [cluster]
root      8542     1  0 17:47 ?        00:00:00 ../redis-server 10.211.55.4:7004 [cluster]
root      8577     1  0 17:48 ?        00:00:00 ../redis-server 10.211.55.4:7005 [cluster]

kill 8436
 
ps -ef|grep redis
root      8462     1  0 17:47 ?        00:00:00 ../redis-server 10.211.55.4:7001 [cluster]
root      8488     1  0 17:47 ?        00:00:00 ../redis-server 10.211.55.4:7002 [cluster]
root      8514     1  0 17:47 ?        00:00:00 ../redis-server 10.211.55.4:7003 [cluster]
root      8542     1  0 17:47 ?        00:00:00 ../redis-server 10.211.55.4:7004 [cluster]
root      8577     1  0 17:48 ?        00:00:00 ../redis-server 10.211.55.4:7005 [cluster]

/opt/redis_cluster/bin/redis-trib.rb check 10.211.55.4:7000
[ERR] Sorry, can't connect to node 10.211.55.4:7000

/opt/redis_cluster/bin/redis-trib.rb check 10.211.55.4:7001
>>> Performing Cluster Check (using node 10.211.55.4:7001)
M: 0b02dc79c42d047a94d2d5bd12cdf3448fe72647 10.211.55.4:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 6f3d4c358463a0e378a5511d7f473e81897e9910 10.211.55.4:7002
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 41e6c6785110b8f9521406e3a698b676a0c68d28 10.211.55.4:7003
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
S: 461f08effa70e742a6f80a597876c11b9f69ebd7 10.211.55.4:7005
   slots: (0 slots) slave
   replicates 6f3d4c358463a0e378a5511d7f473e81897e9910
S: 5968ecb68bf7671bee2a8af201f14280b2dd6042 10.211.55.4:7004
   slots: (0 slots) slave
   replicates 0b02dc79c42d047a94d2d5bd12cdf3448fe72647
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

kill 8514

/opt/redis_cluster/bin/redis-trib.rb check 10.211.55.4:7001
>>> Performing Cluster Check (using node 10.211.55.4:7001)
M: 0b02dc79c42d047a94d2d5bd12cdf3448fe72647 10.211.55.4:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 6f3d4c358463a0e378a5511d7f473e81897e9910 10.211.55.4:7002
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 461f08effa70e742a6f80a597876c11b9f69ebd7 10.211.55.4:7005
   slots: (0 slots) slave
   replicates 6f3d4c358463a0e378a5511d7f473e81897e9910
S: 5968ecb68bf7671bee2a8af201f14280b2dd6042 10.211.55.4:7004
   slots: (0 slots) slave
   replicates 0b02dc79c42d047a94d2d5bd12cdf3448fe72647
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[ERR] Not all 16384 slots are covered by nodes.

# root @ Evan in /opt/redis_cluster/7000 [17:59:33] 
$ ../bin/redis-server ./redis.conf              
8898:C 31 Aug 18:00:03.272 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
8898:C 31 Aug 18:00:03.272 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=8898, just started
8898:C 31 Aug 18:00:03.272 # Configuration loaded

ps -ef|grep redis                         
root      8462     1  0 17:47 ?        00:00:01 ../redis-server 10.211.55.4:7001 [cluster]
root      8488     1  0 17:47 ?        00:00:01 ../redis-server 10.211.55.4:7002 [cluster]
root      8542     1  0 17:47 ?        00:00:01 ../redis-server 10.211.55.4:7004 [cluster]
root      8577     1  0 17:48 ?        00:00:01 ../redis-server 10.211.55.4:7005 [cluster]
root      8899     1  0 18:00 ?        00:00:00 ../redis-server 10.211.55.4:7000 [cluster]

/opt/redis_cluster/bin/redis-trib.rb check 10.211.55.4:7001
>>> Performing Cluster Check (using node 10.211.55.4:7001)
M: 0b02dc79c42d047a94d2d5bd12cdf3448fe72647 10.211.55.4:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 6f3d4c358463a0e378a5511d7f473e81897e9910 10.211.55.4:7002
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: e0c43b03ce8afab1785dff6c83d85a24654e97d3 10.211.55.4:7000
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
S: 461f08effa70e742a6f80a597876c11b9f69ebd7 10.211.55.4:7005
   slots: (0 slots) slave
   replicates 6f3d4c358463a0e378a5511d7f473e81897e9910
S: 5968ecb68bf7671bee2a8af201f14280b2dd6042 10.211.55.4:7004
   slots: (0 slots) slave
   replicates 0b02dc79c42d047a94d2d5bd12cdf3448fe72647
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

Extension

Redis cluster设置密码
　　如果对集群设置密码，那么requirepass和masterauth都需要设置，否则发生主从切换时，就会遇到授权问题。若是使用redis-trib.rb工具构建集群后再设置密码，则集群构建完毕通过config set + config rewrite命令逐个机器设置密码：

config set masterauth passwd123 
config set requirepass passwd123 
config rewrite

(各个节点密码都必须一致，否则Redirected就会失败，推荐这种方式，这种方式会把密码写入到redis.conf里面去，且不用重启)

或在所以redis集群中的redis.conf文件加入：
masterauth passwd123
requirepass passwd123
(这种方式需要重新启动各节点。)

bind 10.211.55.4
daemonize yes

port 7000
dir /opt/redis_cluster/7000/
masterauth passwd123
requirepass passwd123
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

第三种方法是在使用redis-trib.rb工具构建集群前配置masterauth，requirepass(密码必须一直)，配置文件参考第二种方法里给出的示例。重点：修改clinet.rb:

#找出 clinet.rb配置文件
$ find / -name "client.rb"
/usr/local/rvm/src/ruby-2.4.0/gems/xmlrpc-0.2.1/lib/xmlrpc/client.rb
/usr/local/rvm/gems/ruby-2.4.0/gems/redis-4.0.2/lib/redis/client.rb
/usr/local/rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/xmlrpc-0.2.1/lib/xmlrpc/client.rb
#修改配置文件
vim /usr/local/rvm/gems/ruby-2.4.0/gems/redis-4.0.2/lib/redis/client.rb

DEFAULTS = {
      :url => lambda { ENV["REDIS_URL"] },
      :scheme => "redis",
      :host => "127.0.0.1",
      :port => 6379,
      :path => nil,
      :timeout => 5.0,
      :password => nil,
      :db => 0,
      :driver => nil,
      :id => nil,
      :tcp_keepalive => 0,
      :reconnect_attempts => 1,
      :inherit_socket => false
    }

:password => nil, 修改为 :password => 'passwd123',

然后使用redis-trib.rb创建集群
/opt/redis_cluster/bin/redis-trib.rb create --replicas 1 10.211.55.4:7000 10.211.55.4:7001 10.211.55.4:7002 10.211.55.4:7003 10.211.55.4:7004 10.211.55.4:7005

supervisor + Redis cluster

#安装supervisor及编辑配置文件

yum install python-pip
pip install supervisor
echo_supervisord_conf > /etc/supervisord.conf
mkdir /etc/supervisor/ -p

vim /etc/supervisord.conf

[include]
files = /etc/supervisor/*.conf 

#编辑redis之supervisor启动文件

$ cat /etc/supervisor/redis-cluster.conf 
[program:redis0]
command=/opt/redis_cluster/bin/redis-server /opt/redis_cluster/7000/redis.conf
autostart=true
autorestart=true
stdout_logfile=/var/log/redis0.log
stderr_logfile=/var/log/redis0_error.log
stdout_logfile_maxbytes=20MB
stdout_logfile_backups=5
[program:redis1]
command=/opt/redis_cluster/bin/redis-server /opt/redis_cluster/7001/redis.conf
autostart=true
autorestart=true
stdout_logfile=/var/log/redis1.log
stderr_logfile=/var/log/redis1_error.log
stdout_logfile_maxbytes=20MB
stdout_logfile_backups=5
[program:redis2]
command=/opt/redis_cluster/bin/redis-server /opt/redis_cluster/7002/redis.conf
autostart=true
autorestart=true
stdout_logfile=/var/log/redis2.log
stderr_logfile=/var/log/redis2_error.log
stdout_logfile_maxbytes=20MB
stdout_logfile_backups=5
[program:redis3]
command=/opt/redis_cluster/bin/redis-server /opt/redis_cluster/7003/redis.conf
autostart=true
autorestart=true
stdout_logfile=/var/log/redis3.log
stderr_logfile=/var/log/redis3_error.log
stdout_logfile_maxbytes=20MB
stdout_logfile_backups=5
[program:redis4]
command=/opt/redis_cluster/bin/redis-server /opt/redis_cluster/7004/redis.conf
autostart=true
autorestart=true
stdout_logfile=/var/log/redis4.log
stderr_logfile=/var/log/redis4_error.log
stdout_logfile_maxbytes=20MB
stdout_logfile_backups=5
[program:redis5]
command=/opt/redis_cluster/bin/redis-server /opt/redis_cluster/7005/redis.conf
autostart=true
autorestart=true
stdout_logfile=/var/log/redis5.log
stderr_logfile=/var/log/redis5_error.log
stdout_logfile_maxbytes=20MB
stdout_logfile_backups=5

#务必注意
~~~~~~~~~~~~~~~~
redis.conf
daemonize 改为no 
~~~~~~~~~~~~~~~~

#示例
cat /opt/redis_cluster/7001/redis.conf 
----------------------------
bind 10.211.55.4
daemonize no

port 7001
dir /opt/redis_cluster/7001/
masterauth passwd123
requirepass passwd123
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
----------------------------


#启动supervisor

supervisord -c /etc/supervisord.conf

#查询

# root @ Evan in /etc/supervisor [0:48:00] 
$ supervisorctl       
redis0                           RUNNING   pid 15408, uptime 0:00:15
redis1                           RUNNING   pid 15506, uptime 0:00:15
redis2                           RUNNING   pid 15507, uptime 0:00:15
redis3                           RUNNING   pid 15508, uptime 0:00:15
redis4                           RUNNING   pid 15512, uptime 0:00:15
redis5                           RUNNING   pid 15519, uptime 0:00:15
supervisor> exit

#检验Cluster

# root @ Evan in /etc/supervisor [0:48:23] 
$ /opt/redis_cluster/bin/redis-trib.rb check 10.211.55.4:7000
>>> Performing Cluster Check (using node 10.211.55.4:7000)
S: e0c43b03ce8afab1785dff6c83d85a24654e97d3 10.211.55.4:7000
   slots: (0 slots) slave
   replicates 41e6c6785110b8f9521406e3a698b676a0c68d28
M: 5968ecb68bf7671bee2a8af201f14280b2dd6042 10.211.55.4:7004
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: 6f3d4c358463a0e378a5511d7f473e81897e9910 10.211.55.4:7002
   slots: (0 slots) slave
   replicates 461f08effa70e742a6f80a597876c11b9f69ebd7
M: 461f08effa70e742a6f80a597876c11b9f69ebd7 10.211.55.4:7005
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 41e6c6785110b8f9521406e3a698b676a0c68d28 10.211.55.4:7003
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 0b02dc79c42d047a94d2d5bd12cdf3448fe72647 10.211.55.4:7001
   slots: (0 slots) slave
   replicates 5968ecb68bf7671bee2a8af201f14280b2dd6042
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


#systemctl
cat /usr/lib/systemd/system/supervisord.service 

[Unit]
Description=Process Monitoring and Control Daemon
After=rc-local.service nss-user-lookup.target

[Service]
Type=forking
ExecStart=/usr/bin/supervisord -c /etc/supervisord.conf

[Install]
WantedBy=multi-user.target

原文链接

Redis之集群环境搭建
Redis集群一、Redis集群相关概念 1.Redis集群介绍 Redis 集群是一个提供在多个Redis间节...
单机(win10)使用Docker搭建Redis集群（Redis
搭建过程参考：使用Docker搭建Redis集群（Redis Sentinel 集群部署）概述 Redis 集群...
NoSQL Redis 集群架构（Cluster）--Day2
redis集群一、集群redis cluster 1、哨兵的不足 2、redis cluster 3、集群搭建 ...
Redis 集群规范
redis手册Redis 集群规范Redis集群安装Redis集群安装 Redis分布式部署，一致性hash;分布...
centos防止redis crash的配置
redis目录 redis集群练习1(redis cluster tutorial) redis集群练习2 red...
redis集群练习1(redis cluster tutoria
redis目录 redis集群练习1(redis cluster tutorial) redis集群练习2 red...
redis集群练习2
redis目录 redis集群练习1(redis cluster tutorial) redis集群练习2 red...
redis集群练习3
redis目录 redis集群练习1(redis cluster tutorial) redis集群练习2 red...
redis源码阅读
redis源码阅读 1.redis集群客户端API特征支持redis集群:连接redis集群，并发送命令. 支...
2019-07-04 NoSQL-Day02-Redis Clu
Redis Cluster 集群一、集群介绍官方提供的 Redis Cluster 集群方案，很好的解决了集群...

Redis集群

安装Redis

集群模式redis.conf示例

集群所需Redis多实例部署条件

redis-trib创建集群

客户端验证redis集群

使用cluster info命令查看集群状态。###

使用cluster nodes命令查看节点状态。

kill集群所需节点测试

Extension

相关文章

Redis之集群环境搭建

单机(win10)使用Docker搭建Redis集群（Redis

NoSQL Redis 集群架构（Cluster）--Day2

Redis 集群规范

centos防止redis crash的配置

redis集群练习1(redis cluster tutoria

redis集群练习2

redis集群练习3

redis源码阅读

2019-07-04 NoSQL-Day02-Redis Clu

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

程序员