zookeeper架构源码 - 准备工作

前言

---

很早在项目中接触了dubbo，当时使用zookeeper作为dubbo的服务协调、注册中心。一直没有时间去看他的架构，最近有些时间了，打算看一下zookeeper架构和设计思路。

什么是zookeeper

---

ZooKeeper是一个分布式的，开放源码的分布式应用程序协调服务，是Google的Chubby一个开源的实现，是Hadoop和Hbase的重要组件。
它是一个为分布式应用提供一致性服务的软件，提供的功能包括：配置维护、域名服务、分布式同步、组服务等。

zookeeper基础

---

https://www.jianshu.com/p/977726b8d1ca
https://www.jianshu.com/p/0a0c598545fe

docker部署启动zookeeper集群

---

1. 准备zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
# 存放内存数据库快照的位置
dataDir=/data
# 事务日志目录
dataLogDir=/dataLog
clientPort=2181

server.1=zk1:2888:3888
server.2=zk2:2888:3888
server.3=zk3:2888:3888

2. 先编写镜像Dockerfile
   题外话：我根本不想去编写镜像，想直接使用docker默认提供的zookeeper镜像。但是我运行之后发现默认的zookeeper镜像里面居然没有zoo.cfg文件。

FROM zookeeper:3.7.0

MAINTAINER sunpy

COPY ./zoo.cfg /conf/zoo.cfg

EXPOSE 2181

生成镜像：

[root@ecs-140051 zookeeper]# docker build -t zookeeper:sunpy-1 .
Sending build context to Docker daemon  12.29kB
Step 1/4 : FROM zookeeper:3.7.0
 ---> 04ec824d47af
Step 2/4 : MAINTAINER sunpy
 ---> Running in da9be9b11153
Removing intermediate container da9be9b11153
 ---> b6fb30558e24
Step 3/4 : COPY ./zoo.cfg /conf/zoo.cfg
 ---> a8ab3db3935a
Step 4/4 : EXPOSE 2181
 ---> Running in 44af8b104864
Removing intermediate container 44af8b104864
 ---> b489b6f7f157
Successfully built b489b6f7f157
Successfully tagged zookeeper:sunpy-1
[root@ecs-140051 zookeeper]# docker images
REPOSITORY           TAG                   IMAGE ID       CREATED          SIZE
zookeeper            sunpy-1               b489b6f7f157   10 seconds ago   277MB
zookeeper            latest                e3ea652a6eb3   2 days ago       290MB
zookeeper            3.7.0                 04ec824d47af   2 months ago     277MB

3. 使用docker-compose编排部署zookeeper集群

version: '3.7'

services:
  zk1:
    image: zookeeper:sunpy-1
    hostname: zk1
    container_name: zk1
    ports:
      - 3181:2181
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zk1:2888:3888 server.2=zk2:2888:3888 server.3=zk3:2888:3888
    volumes:
      - /home/zookeeper/zk1/data:/data
      - /home/zookeeper/zk1/dataLog:/dataLog

  zk2:
    image: zookeeper:sunpy-1
    hostname: zk2
    container_name: zk2
    ports:
      - 3182:2181
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=zk1:2888:3888 server.2=zk2:2888:3888 server.3=zk3:2888:3888
    volumes:
      - /home/zookeeper/zk2/data:/data
      - /home/zookeeper/zk2/dataLog:/dataLog

  zk3:
    image: zookeeper:sunpy-1
    hostname: zk3
    container_name: zk3
    ports:
      - 3183:2181
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=zk1:2888:3888 server.2=zk2:2888:3888 server.3=zk3:2888:3888
    volumes:
      - /home/zookeeper/zk3/data:/data
      - /home/zookeeper/zk3/dataLog:/dataLog

启动：

[root@ecs-140051 zookeeper]# docker-compose -f zookeeper.yml up -d

查看：

[root@ecs-140051 zookeeper]# docker ps -a
CONTAINER ID   IMAGE                      COMMAND                  CREATED         STATUS         PORTS                                                                     NAMES
fe6bab4099c7   zookeeper:sunpy-1          "/docker-entrypoint.…"   4 seconds ago   Up 3 seconds   2888/tcp, 3888/tcp, 8080/tcp, 0.0.0.0:3183->2181/tcp, :::3183->2181/tcp   zk3
bd746115c284   zookeeper:sunpy-1          "/docker-entrypoint.…"   4 seconds ago   Up 3 seconds   2888/tcp, 3888/tcp, 8080/tcp, 0.0.0.0:3181->2181/tcp, :::3181->2181/tcp   zk1
e451f30a3fab   zookeeper:sunpy-1          "/docker-entrypoint.…"   4 seconds ago   Up 3 seconds   2888/tcp, 3888/tcp, 8080/tcp, 0.0.0.0:3182->2181/tcp, :::3182->2181/tcp   zk2
c0da0a58a8e8   nacos/nacos-server:2.0.3   "bin/docker-startup.…"   8 days ago      Up 16 hours    0.0.0.0:8848->8848/tcp, :::8848->8848/tcp                                 nacos-sunpy
14c6ae3d049c   mysql:latest               "docker-entrypoint.s…"   2 weeks ago     Up 16 hours    33060/tcp, 0.0.0.0:3389->3306/tcp, :::3389->3306/tcp                      mysql

zookeeper启动过程中出现错误：

[root@ecs-140051 zookeeper]# docker logs -f 52badadcd570
2022-08-11 02:55:55,857 [myid:2] - ERROR [main:ZooKeeperServerMain@81] - Unable to access datadir, exiting abnormally
org.apache.zookeeper.server.persistence.FileTxnSnapLog$DatadirException: Unable to create data directory /dataLog/version-2
    at org.apache.zookeeper.server.persistence.FileTxnSnapLog.<init>(FileTxnSnapLog.java:136)
    at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:139)
    at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:113)
    at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:68)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:141)
    at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:91)
Unable to access datadir, exiting abnormally

分析：没有权限访问dataDir，那么肯定不能创建那个/dataLog/version-2文件了。可能是我们挂载数据卷的时候，外部的文件夹没有访问权限。那么我想将当前的文件夹设置一下权限，将权限开放。

[root@ecs-140051 zookeeper]# chmod -R a+w /home/zookeeper/zk1/data
[root@ecs-140051 zookeeper]# chmod -R a+w /home/zookeeper/zk1/dataLog
[root@ecs-140051 zookeeper]# chmod -R a+w /home/zookeeper/zk2/data
[root@ecs-140051 zookeeper]# chmod -R a+w /home/zookeeper/zk2/dataLog
[root@ecs-140051 zookeeper]# chmod -R a+w /home/zookeeper/zk3/data
[root@ecs-140051 zookeeper]# chmod -R a+w /home/zookeeper/zk3/dataLog

结果：问题解决了，zookeeper启动成功了。

远程查看zookeeper集群

---

使用zooInspector查看（为了查看方便，写个bat脚本）：

F:
cd F:\tools\zooInspector\build\
java -jar zookeeper-dev-ZooInspector.jar

架构图

---

• Leader：
  事务请求的唯一调度和处理者，保证集群事务的顺序性，服务的调度者。
  如果任何服务器节点发生故障，该服务器节点将执行自动恢复。
• Follower：
  处理客户端非事务请求，转发事务请求给Leader。
  参与事务请求Proposal提议投票。
  参与Leader选举投票。
• Observer：
  观察zookeeper集群的最新状态变化并将这些状态变更同步过来。
  不参与事务投票。