构建大数据平台（八）redis集群搭建

作者: Mr_ZhaiDK | 来源:发表于2018-03-01 13:41 被阅读0次

一、前提：

（1）集群一般由多个节点组成，节点数量至少6个才能保证组成完整高可用的集群。资源有限，这里使用4台服务器模拟6个节点（用端口区分），如下图：

host	ip	software	port1	port2	system	role
master	192.168.2.158	redis-4.0.8	7001		centos7	m1
slave1	192.168.2.159	redis-4.0.8	7002	7003	centos7	m2,m3
slave2	192.168.2.178	redis-4.0.8	7004		centos7	s1
slave3	192.168.2.179	redis-4.0.8	7005	7006	centos7	s2,s3

（2）Redis使用稳定版本：redis-4.0.8.tar.gz；

（3）每个节点需要开启配置文件中的cluster-enabled yes，让Redis运行在集群模式下。

（4）Redis需要安装依赖库，所有机器都要安装：

yum install gcc gcc-c++ kernel-devel automake autoconf libtool make wget tcl vim psmisc ruby rubygems ruby-devel rpm-build unzip git -y

二、配置单机Redis：

首先要注意在生产环境中目录结构要定义好，防止在项目过多的时候找不到所需的项目

目录统一放在/home下面，首先创建redis项目目录

cd /home

#创建项目目录
mkdir redis
cd /home/redis

将 redis-4.0.8.tar.gz 放在 /home/redis下，并解压缩，解压缩完成后可删除安装包：

#解压缩
tar -zxvf redis-4.0.8.tar.gz

#完成后可删除
rm -f redis-4.0.8.tar.gz

编译安装：

cd /home/redis/redis-4.0.8/
#编译安装
make && make install
#检查一下版本
redis-server  -v

4)启动redis

cd /home/redis/redis-4.0.8/src/
#启动
./redis-server ../redis.conf

5)测试单机redis是否启动成功

新开终端，打开redis客户端

cd /home/redis/redis-4.0.8/src/
#连接redis
./redis-cli
#显示如下：
127.0.0.1：6379>

所有机器都需要安装redis，可以scp复制后再make & make install安装测试

6)单机redis，开机自启动
（1）编辑配置文件，将bind和daemonize改为如下：

bind 0.0.0.0

（2）添加开机启动服务：

vi /etc/systemd/system/redis-server.service

粘贴一下内容

[Unit]
Description=The redis-server Process Manager
After=syslog.target network.target

[Service]
Type=simple
PIDFile=/var/run/redis_6379.pid
ExecStart=/home/redis/redis-4.0.8/src/redis-server /home/redis/redis-4.0.8/redis.conf         
ExecReload=/bin/kill -USR2 $MAINPID
ExecStop=/bin/kill -SIGINT $MAINPID

[Install]
WantedBy=multi-user.target

（3）设置开机启动：

systemctl daemon-reload 
systemctl start redis-server.service 
systemctl enable redis-server.service

（4）检查是否安装成功：

ps -ef|grep redis

三、配置Redis集群：

安装ruby依赖，由于yum命令安装的ruby版本太低，需自行升级，这里采用rvm来更新ruby。
（1）安装RVM:

#安装curl
yum install curl
#建立信任通信
gpg2 --keyserver hkp://keys.gnupg.net --recv-keys D39DC0E3
#下载安装rvm
curl -L get.rvm.io | bash -s stable
#查看安装路径
find / -name rvm -print
#使rvm生效
source /usr/local/rvm/scripts/rvm

（2）查看rvm库中已知的ruby版本

rvm list known
#显示结果如下：
MRI Rubies 
[ruby-]1.8.6[-p420]
[ruby-]1.8.7[-head] # security released on head
[ruby-]1.9.1[-p431]
[ruby-]1.9.2[-p330]
[ruby-]1.9.3[-p551]
[ruby-]2.0.0[-p648]
[ruby-]2.1[.10]
[ruby-]2.2[.7]
[ruby-]2.3[.4]
[ruby-]2.4[.1]
ruby-head

（3）安装一个ruby版本

rvm install 2.4.1

（4）使用一个ruby版本

rvm use 2.4.1

（5）设置默认版本

rvm use 2.4.1 --default

（6）卸载一个已知版本

rvm remove 2.0.0

（7）查看ruby版本:

ruby --version

（8）安装gem redis:

gem install redis

所有机器都需要安装ruby

（4）检查redis-trib.rb的可用性：

cd /home/redis/redis-4.0.8/src/
./redis-trib.rb

#出现如下，则代表可用
Usage: redis-trib <command> <options> <arguments ...>

  create          host1:port1 ... hostN:portN
                  --replicas <arg>
  check           host:port
  info            host:port
  fix             host:port
                  --timeout <arg>
  reshard         host:port
                  --from <arg>
                  --to <arg>
                  --slots <arg>
                  --yes
                  --timeout <arg>
                  --pipeline <arg>
  rebalance       host:port
                  --weight <arg>
                  --auto-weights
                  --use-empty-masters
                  --timeout <arg>
                  --simulate
                  --pipeline <arg>
                  --threshold <arg>
  add-node        new_host:new_port existing_host:existing_port
                  --slave
                  --master-id <arg>
  del-node        host:port node_id
  set-timeout     host:port milliseconds
  call            host:port command arg arg .. arg
  import          host:port
                  --from <arg>
                  --copy
                  --replace
  help            (show this help)

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

配置每个节点

（1）在master上：

#在/home/redis/目录下新建一个redis-cluster文件夹
cd /home/redis/
mkdir -p redis-cluster

#新建1个文件夹，用来区分节点
cd /home/redis/redis-cluster
mkdir -p  7001

#复制配置文件到新文件夹下
cp /home/redis/redis-4.0.8/redis.conf /home/redis/redis-cluster/7001/redis.conf

#编辑配置文件
vi /home/redis/redis-cluster/7001/redis.conf

必要需要修改如下，其他细化优化的配置项可暂时保持默认

bind master　　     #本机IP
port 7001　　　   　　     #改为设定的端口
pidfile /var/run/redis_7001.pid              #pid存放地址
daemonize yes　             #后台启动
cluster-enabled yes　　　　　 　　　 #启动集群模式
cluster-config-file nodes-7001.conf　　#集群内部配置文件，改掉端口号　
cluster-node-timeout 15000　　　　　#节点超时时间，单位：毫秒

（2）在slave1上：

#新建1个文件夹，用来区分节点
cd /home/redis/redis-cluster
mkdir -p  7002
mkdir -p  7003

#......其他操作，参考master......

（3）在slave2上：

#新建1个文件夹，用来区分节点
cd /home/redis/redis-cluster
mkdir -p  7004

#......其他操作，参考master......

（4）在slave3上：

#新建1个文件夹，用来区分节点
cd /home/redis/redis-cluster
mkdir -p  7005
mkdir -p  7006

#......其他操作，参考master......

（5）启动节点

#master
redis-server /home/redis/redis-cluster/7001/redis.conf
#slave1
redis-server /home/redis/redis-cluster/7002/redis.conf
redis-server /home/redis/redis-cluster/7003/redis.conf
#slave2
redis-server /home/redis/redis-cluster/7004/redis.conf
#slave3
redis-server /home/redis/redis-cluster/7005/redis.conf
redis-server /home/redis/redis-cluster/7006/redis.conf

#查看redis状态
ps -ef|grep redis

#停止服务
redis-cli -h master -p 7001 shutdown
redis-cli -h slave1 -p 7002 shutdown
redis-cli -h slave1 -p 7003 shutdown
redis-cli -h slave2 -p 7004 shutdown
redis-cli -h slave3 -p 7005 shutdown
redis-cli -h slave3 -p 7006 shutdown

（6）创建集群

#在master上运行（使用ip ：port）
cd /home/redis/redis-4.0.8/src/
./redis-trib.rb create  --replicas 1 192.168.2.158:7001 192.168.2.159:7002 192.168.2.159:7003 192.168.2.177:7004 192.168.2.178:7005 192.168.2.178:7006

如果出现下述错误：

>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.2.158:7001
192.168.2.159:7002
192.168.2.177:7004
Adding replica 192.168.2.159:7003 to 192.168.2.158:7001
Adding replica 192.168.2.178:7006 to 192.168.2.159:7002
Adding replica 192.168.2.178:7005 to 192.168.2.177:7004
M: d3ca605d95094a7699418de2520312e509cc4c8a 192.168.2.158:7001
   slots:0-5460 (5461 slots) master
M: bbc9da22c5a9fcaad9b99ce60b8503c0d91091e9 192.168.2.159:7002
   slots:5461-10922 (5462 slots) master
S: 7ea311b36f527d0a5b95cf5e2629e0ad9e56047c 192.168.2.159:7003
   replicates d3ca605d95094a7699418de2520312e509cc4c8a
M: 43ceef9949a28548c6f1fa66a75ddce3c33a2374 192.168.2.177:7004
   slots:10923-16383 (5461 slots) master
S: 0214320aeddb789abc452c48871bcc9ee02fd7a7 192.168.2.178:7005
   replicates 43ceef9949a28548c6f1fa66a75ddce3c33a2374
S: b39f1ac1c78a77ccf656a5bf5fc8c0cebd01a787 192.168.2.178:7006
   replicates bbc9da22c5a9fcaad9b99ce60b8503c0d91091e9
Can I set the above configuration? (type 'yes' to accept): yes
/usr/local/rvm/gems/ruby-2.4.1/gems/redis-4.0.1/lib/redis/client.rb:119:in `call': ERR Slot 0 is already busy (Redis::CommandError)
    from /usr/local/rvm/gems/ruby-2.4.1/gems/redis-4.0.1/lib/redis.rb:2764:in `block in method_missing'
    from /usr/local/rvm/gems/ruby-2.4.1/gems/redis-4.0.1/lib/redis.rb:45:in `block in synchronize'
    from /usr/local/rvm/rubies/ruby-2.4.1/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
    from /usr/local/rvm/gems/ruby-2.4.1/gems/redis-4.0.1/lib/redis.rb:45:in `synchronize'
    from /usr/local/rvm/gems/ruby-2.4.1/gems/redis-4.0.1/lib/redis.rb:2763:in `method_missing'
    from ./redis-trib.rb:212:in `flush_node_config'
    from ./redis-trib.rb:906:in `block in flush_nodes_config'
    from ./redis-trib.rb:905:in `each'
    from ./redis-trib.rb:905:in `flush_nodes_config'
    from ./redis-trib.rb:1426:in `create_cluster_cmd'
    from ./redis-trib.rb:1830:in `<main>'

解决方法如下：
（a）删除所有节点得

#1.删除dump.rdb
cd /home/redis/redis-4.0.8
rm -f dump.rdb
#2.删除所有节点得node-700x.conf文件

（b）运行以下代码：

redis-cli -p 7001 -h master FLUSHALL
redis-cli -p 7001 -h master CLUSTER RESET SOFT
redis-cli -p 7002 -h slave1 FLUSHALL
redis-cli -p 7002 -h slave1 CLUSTER RESET SOFT
redis-cli -p 7003 -h slave1 FLUSHALL
redis-cli -p 7003 -h slave1 CLUSTER RESET SOFT
redis-cli -p 7004 -h slave2 FLUSHALL
redis-cli -p 7004 -h slave2 CLUSTER RESET SOFT
redis-cli -p 7005 -h slave3 FLUSHALL
redis-cli -p 7005 -h slave3 CLUSTER RESET SOFT
redis-cli -p 7006 -h slave3 FLUSHALL
redis-cli -p 7006 -h slave3 CLUSTER RESET SOFT

（c）重启redis服务，最后启动集群，正确结果如下：

>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.2.158:7001
192.168.2.159:7002
192.168.2.177:7004
Adding replica 192.168.2.159:7003 to 192.168.2.158:7001
Adding replica 192.168.2.178:7006 to 192.168.2.159:7002
Adding replica 192.168.2.178:7005 to 192.168.2.177:7004
M: 3198f98b2a6df119ae66dc98cf2e13ce9f70e1e6 192.168.2.158:7001
   slots:0-5460 (5461 slots) master
M: b24938d9fd4f79e7a27ba6c532d1333230089859 192.168.2.159:7002
   slots:5461-10922 (5462 slots) master
S: 0b97a68fc49b9e4c5ee21ef4428bba98bac13e0f 192.168.2.159:7003
   replicates 3198f98b2a6df119ae66dc98cf2e13ce9f70e1e6
M: 9ce4c626e9552ea3d1ceeec998994a85e6379ddf 192.168.2.177:7004
   slots:10923-16383 (5461 slots) master
S: 62a75986d63859a6af3a307548b76cd43f1ecf56 192.168.2.178:7005
   replicates 9ce4c626e9552ea3d1ceeec998994a85e6379ddf
S: b39f1ac1c78a77ccf656a5bf5fc8c0cebd01a787 192.168.2.178:7006
   replicates b24938d9fd4f79e7a27ba6c532d1333230089859
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.
>>> Performing Cluster Check (using node 192.168.2.158:7001)
M: 3198f98b2a6df119ae66dc98cf2e13ce9f70e1e6 192.168.2.158:7001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 62a75986d63859a6af3a307548b76cd43f1ecf56 192.168.2.178:7005
   slots: (0 slots) slave
   replicates 9ce4c626e9552ea3d1ceeec998994a85e6379ddf
S: b39f1ac1c78a77ccf656a5bf5fc8c0cebd01a787 192.168.2.178:7006
   slots: (0 slots) slave
   replicates b24938d9fd4f79e7a27ba6c532d1333230089859
M: 9ce4c626e9552ea3d1ceeec998994a85e6379ddf 192.168.2.177:7004
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: b24938d9fd4f79e7a27ba6c532d1333230089859 192.168.2.159:7002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: 0b97a68fc49b9e4c5ee21ef4428bba98bac13e0f 192.168.2.159:7003
   slots: (0 slots) slave
   replicates 3198f98b2a6df119ae66dc98cf2e13ce9f70e1e6
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

测试集群
（1）在master上，创建数据，运行以下代码：

cd /home/redis/redis-4.0.8/src/
#连接redis
./redis-cli -h 192.168.2.158 -c -p 7001
192.168.2.158:7001> set title test
OK
192.168.2.158:7001> get title
"test"
192.168.2.158:7001>

（2）退出后，连接其他终端，进行测试：

./redis-cli -h 192.168.2.178 -c -p 7006
192.168.2.178:7006> get title
-> Redirected to slot [2217] located at 192.168.2.158:7001
"test"
192.168.2.158:7001>

至此，redis集群搭建完毕

构建大数据平台（八）redis集群搭建

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读