CentOS中Redis-Cluster集群配置

我是三台机器要做cluster集群，所以每台机器启2个端口。
服务器1：192.168.31.146
服务器2：192.168.31.147
服务器3：192.168.31.148
三台所用端口均为 7001、7002

官方推荐RedisCluster最少6个节点，三主三从。并且物理节点数要为奇数，可根据(n-1)/2算出来你最多可挂几个物理节点保证不影响正常运行

1. 安装集群环境
   1.1 安装gcc,g++
   yum install gcc g++
   1.2 安装ruby脚本运行环境
   yum install ruby
   1.3 修改ruby下载源，不然国外的源下载不了
   gem sources --add https://gems.ruby-china.org/ --remove https://rubygems.org/
   1.4 安装redis集群工具
   gem install redis -v 3.3.5

2. 安装redis
   2.1 cd /下载redis、解压
   wget http://download.redis.io/releases/redis-3.2.4.tar.gz
tar -zxvf redis-3.2.4.tar.gz
   2.3 编译安装到指定目录
   cd redis-3.2.4
make install PREFIX=/usr/local/redis
   2.4 从redis的源码目录中复制redis.conf到redis的安装目录
   cp redis.conf /usr/local/redis/bin
   2.5 跳转到安装目录
   cd /usr/local/redis/bin

3. 集群的搭建
   现在才是正真的集群搭建。每台服务器创建2个实例，端口7001～7002
   3.1 跳到指定目录、创建目录redis-cluster
   cd /usr/local
mkdir redis-cluster
   3.2 进入redis/bin目录，把持久化文件删掉
   cd redis/bin
rm -f dump.rdb
   3.3 创建多个redis实例
   cd ..
cp -r bin ../redis-cluster/redis01
cp -r bin ../redis-cluster/redis02
   3.5 退出并进入redis-cluster目录
   cd ../redis-cluster
   3.6 去编辑各个实例下面的配置文件
   vi redis01/redis.conf

   修改1: port端口号(不要重复，依次修改）
   修改2: bind 的IP 127.0.0.1 改为 0.0.0.0
   修改3: 打开cluster-enable前面的注释
   修改4: daemonize 改为yes
   修改5: maxmemory 5gb （Redis最大使用的内存大小）
   修改6: maxmemory-policy allkeys-lru （内存满了使用LRU算法清除key）
   

   同理，去 redis02 修改这些配置

   3.7 把创建集群的ruby脚本复制到redis-cluster的目录下
   去到redis的安装文件
   cd /usr/local/redis-3.2.4/src
   复制脚本到redis-cluster目录下
   cp *.rb /usr/local/redis-cluster/
   3.8 再回去redis-cluster目录
   cd /usr/local/redis-cluster

4. 开始创建集群
   这个时候开始创建集群，但是需要把每一个实例都启动起来好麻烦，所有在这时候创建一个脚本
   4.1 创建redis集群启动的脚本
   vim startall.sh
   内容如下：

   #!/bin/sh
   # chkconfig: 2345 90 10
   # description: Redis is a persistent key-value database
   
   cd /usr/local/redis-cluster/redis01
   /usr/local/redis-cluster/redis01/redis-server  redis.conf
   cd  ..
   cd /usr/local/redis-cluster/redis02
   /usr/local/redis-cluster/redis02/redis-server  redis.conf
   cd  ..
   

   编辑脚本权限
   chmod +x startall.sh
   利用脚本启动redis集群
   ./startall.sh
   查看他们的运行状态
   ps aux|grep redis

5. 创建集群
   5.1 我们还缺少一个东西没有安装（如果已安装请忽略）
   gem install redis -v 3.3.5

   这里注意，我指定版本是因为我的Ruby是2.0.0版本的，最新版本redis 4.0.0 是要求Ruby最低2.2.0版本以上的

   5.2 创建集群

   ./redis-trib.rb create --replicas 1 192.168.31.146:7001 192.168.31.146:7002 192.168.31.147:7001 192.168.31.147:7002  192.168.31.148:7001 192.168.31.148:7002 
   

---

---

这里需要注意，以上配置你需要在两台或两台以上的机器内都要配置，并且保证各个机器上的redis服务和端口是启动状态，这样才可以成功创建集群。

---

---

#查看集群状态
/usr/local/redis/bin/redis-cli -c -p 7001 cluster nodes

成功视图

6. 开机自启动redis-cluster

# 拷贝脚本到启动目录
cp /usr/local/redis-cluster/startall.sh /etc/init.d/redisd

# 添加启动配置
chkconfig redisd on

如果要扩展新的机器节点参考
提醒一点，新加入集群的master节点需要重新slots，把节点插到分配的hash槽内才能使用参考

如果碰巧所有redis集群内的服务器都同时重启了，那么需要重新创建集群。
那么你需要把所有redis服务停掉
ps -ef | grep redis | awk '{print $2}' | xargs kill
并删除各个redis-cluster目录下各个端口文件夹里的dump.rdb 和 nodes.conf文件，如：

示例

下面是我以前的配置备份，作为参考

[root@localhost redis-stable]# yum install net-tools -y
[root@localhost script]# yum install wget -y
[root@localhost script]# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
[root@localhost script]# yum install epel-release-latest-7.noarch.rpm
[root@localhost script]# yum repolist
[root@localhost script]# yum install -y gcc tcl
[root@localhost script]# mkdir /soft
[root@localhost script]# cd /soft
[root@localhost script]# wget http://download.redis.io/releases/redis-stable.tar.gz
[root@localhost script]# tar zxf redis-stable.tar.gz -C /usr/local/src/
[root@localhost script]# cd /usr/local/src/redis-stable/
[root@localhost script]# mkdir /usr/local/redis3
[root@localhost script]# make PREFIX=/usr/local/redis3 install
[root@localhost script]# mkdir -p /usr/local/redis3/cluster
[root@localhost script]# cp /usr/local/src/redis-stable/redis.conf /usr/local/redis3/cluster/redis-cluster.conf
[root@localhost script]# mkdir -p /data/redis/{data,log,var}
[root@localhost script]# vim /usr/local/redis3/cluster/redis-cluster.conf
daemonize yes
pidfile /data/redis/var/redis.pid
bind 0.0.0.0
port 7000
unixsocket /data/redis/var/redis.sock
timeout 0
tcp-keepalive 0
databases 16
cluster-enabled yes
cluster-config-file /usr/local/redis3/cluster/nodes.conf
cluster-node-timeout 15000
cluster-migration-barrier 1
cluster-require-full-coverage yes
loglevel warning
logfile /data/redis/log/redis.log
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /data/redis/data
maxmemory 20000mb
appendonly no
appendfsync everysec
no-appendfsync-on-rewrite yes
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 1024
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
activerehashing yes

[root@localhost redis-stable]# yum install -y ruby rubygems  
[root@localhost redis-stable]# gem sources --remove https://rubygems.org/
[root@localhost redis-stable]# gem sources -a https://ruby.taobao.org/
[root@localhost redis-stable]# gem sources -l
[root@localhost redis-stable]# gem install redis
[root@localhost redis-stable]# cp /usr/local/src/redis-stable/src/redis-trib.rb /usr/local/redis3/bin/redis-trib
[root@localhost redis-stable]# cat >> /etc/hosts << EOF
172.17.25.120 redisCluster1.dev.inc.ubaby.cn redisCluster1
172.17.25.130 redisCluster2.dev.inc.ubaby.cn redisCluster2
172.17.25.121 redisCluster3.dev.inc.ubaby.cn redisCluster3
172.17.25.131 redisCluster4.dev.inc.ubaby.cn redisCluster4
172.17.25.122 redisCluster5.dev.inc.ubaby.cn redisCluster5
172.17.25.132 redisCluster6.dev.inc.ubaby.cn redisCluster6
EOF
[root@localhost redis-stable]# /usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/redis-cluster.conf
[root@localhost redis-stable]# /usr/local/redis3/bin/redis-trib create --replicas 1 172.17.25.120:7000 172.17.25.121:7000 172.17.25.122:7000 172.17.25.130:7000 172.17.25.131:7000 172.17.25.132:7000
[root@localhost redis-stable]# /usr/local/redis3/bin/redis-cli -p 7000 cluster nodes

重新配置，清空内容（如下）

rm -rf nodes_7000.conf *
rm -rf /data/redis/data_7000/*
rm -rf /data/redis/data_7001/*
rm -rf /data/redis/data_7002/*
rm -rf /data/redis/data_7003/*
rm -rf /data/redis/data_7004/*
rm -rf /data/redis/data_7005/*
ps -ef | grep redis | awk '{print $2}' | xargs kill
ps -ef | grep redis
ls
../bin/redis-server redis_7000.conf
ps -ef | grep redis
ls
ll -Z
chmod 777 ./*
../bin/redis-server redis_7000.conf
ps -ef | grep redis
ls /data/redis/data_7000/
vi redis_7000.conf
vi redis_7001.conf
vi redis_7002.conf
vi redis_7003.conf
vi redis_7004.conf
vi redis_7005.conf
../bin/redis-server redis_7000.conf
../bin/redis-server redis_7001.conf
../bin/redis-server redis_7002.conf
../bin/redis-server redis_7003.conf
../bin/redis-server redis_7004.conf
../bin/redis-server redis_7005.conf
 /usr/local/redis3/bin/redis-trib create --replicas 1 192.168.1.60:7000 192.168.1.60:7001 192.168.1.60:7002 192.168.1.60:7003 192.168.1.60:7004 192.168.1.60:7005 

systemctl stop firewalld.service #关闭防火墙
/usr/local/redis3/bin/redis-cli -c -p 7000   
127.0.0.1:7000> config set protected-mode "no"  #关闭保护模式

一. 问题如下

在192.168.56.57客户端登录192.168.56.56的redis服务器时，报错如下：  
[root@localhost src]# ./redis-cli -h  192.168.56.56  -p  6379 -a  "aabbcc"  
192.168.56.56:6379> ping  
Error: Connection reset by peer  

再telnet一下192.168.56.56的redis服务器的6379端口,提示redis服务有保护模式,需要解除  
[root@localhost src]# telnet 192.168.56.56  6379  
Trying 192.168.56.56...  
Connected to 192.168.56.56.  
Escape character is '^]'.  

-DENIED Redis is running in protected modebecause protected mode is enabled, no bind address was specified, noauthentication password is requested to clients.   
In this mode connections areonly accepted from the loopback interface. If you want to connect from externalcomputers to Redis you may adopt one of the following   
solutions: 1) Justdisable protected mode sending the command 'CONFIG SET protected-mode no' fromthe loopback interface by connecting to Redis from the same host   
the server isrunning, however MAKE SURE Redis is not publicly accessible from internet ifyou do so. Use CONFIG REWRITE to make this change permanent.   
2) Alternativelyyou can just disable the protected mode by editing the Redis configurationfile, and setting the protected mode option to 'no', and then restarting theserver.   
3) If you started the server manually just for testing, restart it withthe '--protected-mode no' option.   
4) Setup a bind address or an authenticationpassword. NOTE: You only need to do one of the above things in order for theserver to start accepting connections from the outside.  
Connection closed by foreign host.  

二. 解决方案

1、修改redis服务器的配置文件  
vi redis.conf    
  
注释以下绑定的主机地址  
# bind 127.0.0.1  
   
2、修改redis服务器的参数配置  
  
修改redis的守护进程为no ，不启用  
127.0.0.1:6379> config set daemonize "no"  
OK  
  
修改redis的保护模式为no，不启用  
127.0.0.1:6379> config set protected-mode "no"  
OK  

三. 问题解决

再次telnet一下192.168.56.56的redis服务器的6379端口，无问题  
[root@localhost Packages]# telnet 192.168.56.56  6379  
Trying 192.168.56.56...  
Connected to 192.168.56.56.  
Escape character is '^]'.  
  
  
再次在192.168.56.57客户端登录192.168.56.56的redis服务器，无问题  
[root@localhost src]# ./redis-cli -h  192.168.56.56 -p 6379 -a "aabbcc"  
192.168.56.56:6379> ping  
PONG