GlusterFS部署

GlusterFS部署

准备

1. 目标

安装GlusterFS集群的主要目的是为k8s集群提供分布式持久化存储。

GlusterFS部署使用3台服务器，服务名称与IP如下

/application/ 为单独的分区

/dev/mapper/centos-application  560G  1.3G  558G   1% /application

	name	ip
1	k8s-server-1	192.168.0.162
2	k8s-server-2	192.168.0.165
3	k8s-server-3	192.168.0.166

2. 环境准备

• 关闭防火墙

• 关闭seliux

集群安装

1. 准备repo源

在所有服务器上执行

# cat >/etc/yum.repos.d/gluster-epel.repo << EOF
[gluster]
name=gluster
baseurl=https://buildlogs.centos.org/centos/7/storage/x86_64/gluster-3.8/
gpgcheck=0
enabled=1
EOF

2. 准备数据目录

在所有服务器上执行

mkdir -p /application/data/brick1
ln -s /application/data/ /data

3. 服务安装与启动

yum install glusterfs-server
systemctl start   glusterd
systemctl enable   glusterd   
systemctl status  glusterd

4. 集群认证

在k8s-server-1上执行

gluster peer probe k8s-server-2
gluster peer probe k8s-server-3

5. 查看集群集群状态

### 服务器k8s-server-1
# gluster peer status
Number of Peers: 2

Hostname: k8s-server-2
Uuid: 7df14213-28e2-4764-a709-51700b8846ba
State: Peer in Cluster (Connected)

Hostname: k8s-server-3
Uuid: 001879a8-9528-4baa-99cc-50e83d562327
State: Peer in Cluster (Connected)

### 服务器k8s-server-2
# gluster peer status
Number of Peers: 2

Hostname: k8s-server-1
Uuid: bee8fb5e-1297-434f-a0ed-42f9c9e673c7
State: Peer in Cluster (Connected)

Hostname: k8s-server-3
Uuid: 001879a8-9528-4baa-99cc-50e83d562327
State: Peer in Cluster (Connected)

### 服务器k8s-server-3
# gluster peer status
Number of Peers: 2

Hostname: k8s-server-1
Uuid: bee8fb5e-1297-434f-a0ed-42f9c9e673c7
State: Peer in Cluster (Connected)

Hostname: k8s-server-2
Uuid: 7df14213-28e2-4764-a709-51700b8846ba
State: Peer in Cluster (Connected)

6. 创建GlusterFS volume

在所有服务器上执行

 mkdir -p /data/brick1/gv0

在k8s-server-1上执行

gluster volume create gv0 replica 3 k8s-server-1:/data/brick1/gv0 k8s-server-2:/data/brick1/gv0 k8s-server-3:/data/brick1/gv0
gluster volume start gv0

查看卷空间

#  gluster volume info
Volume Name: gv0
Type: Replicate
Volume ID: e64334b8-61aa-49e9-b310-351bcaf58412
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: k8s-server-1:/data/brick1/gv0
Brick2: k8s-server-2:/data/brick1/gv0
Brick3: k8s-server-3:/data/brick1/gv0
Options Reconfigured:
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on

Gluster 测试

mount -t glusterfs k8s-server-1:/gv0 /mnt 
# 在/mnt下创建100个文件
for i in `seq -w 1 100`; do cp -rp /var/log/messages /mnt/copy-test-$i; done

到/data/brick1/gv0 下查看，可以看到文件