《分布式_Zookeeper》_Zookeeper核心概念以及原

作者: tjhuey | 来源:发表于2018-11-18 18:29 被阅读65次

学习任何一个东西，系统的去明白其方法论是有必要的！！

官方定义

“ZooKeeper is a centralized service for maintaining configuration
information, naming, providing distributed synchronization, and providing
group services. All of these kinds of services are used in some form or
another by distributed applications. Each time they are implemented there is
a lot of work that goes into fixing the bugs and race conditions that are
inevitable. Because of the difficulty of implementing these kinds of services,
applications initially usually skimp on them ,which make them brittle in the
presence of change and difficult to manage. Even when done correctly,
different implementations of these services lead to management complexity
when the applications are deployed.”
简述功能: 高可用、高性能且一致的开源协调服务，它提供了一项基本服务：统一命名服务、、布式协调、存储数据、监听与通知等功能
（链接：http://zookeeper.apache.org）

核心架构图

jiagou1.png

jiagou2.png
（链接：http://zookeeper.apache.org/doc/current/zookeeperOver.html）

zoo.cfg配置

zoo.cfg.png

涉及角色

Leader：
Leader作为整个ZooKeeper集群的主节点，负责响应所有对ZooKeeper状态变更的请求。它会将每个状态更新请求进行排序和编号，以便保证整个集群内部消息处理的FIFO，写操作都走leader，zk里面leader只有一个
Follower ：
Follower的逻辑就比较简单了。除了响应本服务器上的读请求外，follower还要处理leader的提议，并在leader提交该提议时在本地也进行提交。另外需要注意的是，leader和follower构成ZooKeeper集群的法定人数，也就是说，只有他们才参与新leader的选举、响应leader的提议。 帮助leader处理读请求，投票权
Observer ：
如果ZooKeeper集群的读取负载很高，或者客户端多到跨机房，可以设置一些observer服务器，以提高读取的吞吐量。Observer和Follower比较相似，只有一些小区别：首先observer不属于法定人数，即不参加选举也不响应提议；其次是observer不需要将事务持久化到磁盘，一旦observer被重启，需要从leader重新同步整个名字空间。 没有投票权利,可以处理读请求

特性

Zookeeper 是一个由多个 server 组成的集群
一个 leader，多个 follower
每个 server 保存一份数据副本
全局数据一致
分布式读 follower，写由 leader 实施
更新请求转发，由 leader 实施
更新请求顺序进行，来自同一个 client 的更新请求按其发送顺序依次执行
数据更新原子性，一次数据更新要么成功，要么失败
全局唯一数据视图，client 无论连接到哪个 server，数据视图都是一致的
实时性，在一定事件范围内，client 能读到最新数据

Znode（`org.apache.zookeeper.data.Stat implements Record` 数据结构参考）

ZooKeeper操作和维护的为一个个数据节点，称为 znode，采用类似文件系统的层级树状结构进行管理。如果 znode 节点包含数据则存储为字节数组（byte array）。创建 znode 时需要指定节点类型znode 共有 4 种类型，分别为：持久（无序）、临时（无序）、持久有序和临时有序

PERSISTENT 持久（无序），如果不手动删除 是一直存在的
PERSISTENT_SEQUENTIAL  持久（有序）
EPHEMERAL 临时（无序） 客户端session失效就会随着删除节点 没有子节点
EPHEMERAL_SEQUENTIAL 临时有序 自增

Watcher (org.apache.zookeeper.watcher 数据结构参考)

Watcher（事件监听器），是Zookeeper中的一个很重要的特性。Zookeeper允许用户在指定节点上注册一些Watcher，并且在一些特定事件触发的时候，ZooKeeper服务端会将事件通知到感兴趣的客户端上去，该机制是Zookeeper实现分布式协调服务的重要特性。

ACL（`org.apache.zookeeper.ZooDefs` 数据结构参考）

ACL(Access Control List)
内置的 ACL schemes：
world：默认方式，相当于全世界都能访问
auth：代表已经认证通过的用户(cli中可以通过addauth digest user:pwd 来添加当前上下文中的授权用户)
digest：即用户名:密码这种方式认证，这也是业务系统中最常用的
ip：使用Ip地址认证
ACL支持权限：
CREATE: 能创建子节点
READ：能获取节点数据和列出其子节点
WRITE: 能设置节点数据
DELETE: 能删除子节点
ADMIN: 能设置权限

其他

高性能 ZooKeeper 是高性能的。在“读”多于“写”的应用程序中尤其地高性能，因为“写”会导致所有的服务器间同步状态。（“读”多于“写”是协调服务的典型场景。）顺序访问对于来自客户端的每个更新请求，ZooKeeper 都会分配一个全局唯一的递增编号，这个编号反应了所有事务操作的先后顺序，应用程序可以使用 ZooKeeper 这个特性来实现更高层次的同步原语。这个编号也叫做时间戳——zxid（Zookeeper Transaction Id）

Java 原生API(`假设zk集群环境以及搭好并有效启动`)

1.引入maven依赖

        <dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.4.9</version>
        </dependency>

2.Znode相关

package com.huey.znode;
/**
 * @author huey China.
 * @Description : ZNode CRUD
 * @Date Created in 2018/11/18 下午3:31
 */

import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;

import java.io.IOException;

public class ZookeeperCrud {
    private String connectString = "192.168.59.2:2181,192.168.59.3:2181,192.168.59.4:2181";

    private ZooKeeper zookeeper;

    public ZookeeperCrud() {
        try {
            zookeeper = new ZooKeeper(connectString, 5000, null);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }


    public String createPersistent(String path, String data) {
        try {
            return zookeeper.create(path, data.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        } catch (KeeperException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return null;
    }


    public String createEphemeral(String path, String data) {
        try {
            return zookeeper.create(path, data.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
        } catch (KeeperException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return null;
    }


    /***
     * 获取信息
     * @param path
     * @return
     * @throws KeeperException
     * @throws InterruptedException
     */
    public String getData(String path) throws KeeperException, InterruptedException {
        byte data[] = zookeeper.getData(path, false, null);
        data = (data == null) ? "null".getBytes() : data;
        return new String(data);
    }


    /***
     * 更新信息
     * @param path
     * @param data
     * @return
     * @throws KeeperException
     * @throws InterruptedException
     */
    public Stat setData(String path, String data) throws KeeperException, InterruptedException {
        return zookeeper.setData(path, data.getBytes(), -1);
    }

    /***
     * 是否存在
     * @param path
     * @return
     * @throws KeeperException
     * @throws InterruptedException
     */
    public Stat exists(String path) throws KeeperException, InterruptedException {
        return zookeeper.exists(path, false);

    }


    /***
     * 删除
     * @param path
     * @return
     * @throws KeeperException
     * @throws InterruptedException
     */
    public void delete(String path) throws KeeperException, InterruptedException {
        zookeeper.delete(path, -1);
    }

    /***
     * 删除 递归
     * @param path
     * @return
     * @throws KeeperException
     * @throws InterruptedException
     */
    public void deleteRecursive(String path) throws KeeperException, InterruptedException {
        ZKUtil.deleteRecursive(zookeeper, path);
    }

    public void close() throws InterruptedException {
        zookeeper.close();
    }
}

package com.huey.znode;
/**
 * @author huey China.
 * @Description : ZNode Test
 * @Date Created in 2018/11/18 下午3:31
 */

import org.apache.zookeeper.KeeperException;

public class ZookeeperCrudTest {
    public static void main(String[] args) throws KeeperException, InterruptedException {
        ZookeeperCrud zookeeperCrud = new ZookeeperCrud();

        //定义目录
        String newDir = "/huey_key2";

        String data = "";
        data = zookeeperCrud.getData(newDir);
        System.out.println("This time dir " + newDir + "value  is " + data);
        //删
        zookeeperCrud.delete(newDir);
        // 是否存在 ok
        if (null == zookeeperCrud.exists(newDir)) {
            System.out.println("This time is not have this dir");
            // 增
            zookeeperCrud.createPersistent(newDir, "newDir value");
            // 查

            data = zookeeperCrud.getData(newDir);
            System.out.println("This time dir " + newDir + "value  is " + data);

            // 改
            zookeeperCrud.setData(newDir, "new value");
        }


    }
}

3.Watch相关

package com.huey.watcher;
/**
* @author huey China.
* @Description : zk watcher
* @Date Created in 2018/11/18 下午3:48
*/
import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;

public class ZookeeperWatcher2 implements Watcher {
   private String connectString="192.168.59.2:2181,192.168.59.3:2181,192.168.59.4:2181";

   private ZooKeeper zookeeper;

   public ZookeeperWatcher2() {
      try {
         this.zookeeper = new ZooKeeper(connectString, 5000,this);
      } catch (Exception e) {
         e.printStackTrace();
      }
   }


   /***
    * 创建持久节点
    * @param path
    * @param data
    * @return
    */
   public String createPersistent(String path,String data){
      try {
         return  zookeeper.create(path, data.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
      } catch (KeeperException e) {
         e.printStackTrace();
      } catch (InterruptedException e) {
         e.printStackTrace();
      }
      return  null;
   }


   /***
    * 创建临时节点
    * @param path
    * @param data
    * @return
    */
   public String createEphemeral(String path,String data){
      try {
         return  zookeeper.create(path, data.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
      } catch (KeeperException e) {
         e.printStackTrace();
      } catch (InterruptedException e) {
         e.printStackTrace();
      }
      return  null;
   }

   /***
    * 更新信息
    * @param path
    * @return
    * @throws KeeperException
    * @throws InterruptedException
    */
   public String getData(String path,boolean watcher) throws KeeperException, InterruptedException {
      byte data[] = zookeeper.getData(path,watcher,null);
      data = (data == null)? "null".getBytes() : data;
      return new String(data);
   }


   /***
    * 更新信息
    * @param path
    * @param data
    * @return
    * @throws KeeperException
    * @throws InterruptedException
    */
   public Stat setData(String path, String data) throws KeeperException, InterruptedException {
      return zookeeper.setData(path, data.getBytes(), -1);
   }

   /***
    * 是否存在
    * @param path
    * @return
    * @throws KeeperException
    * @throws InterruptedException
    */
   public Stat exists(String path, boolean watcher) throws KeeperException, InterruptedException {
      return zookeeper.exists(path,watcher);

   }


   /***
    * 删除
    * @param path
    * @return
    * @throws KeeperException
    * @throws InterruptedException
    */
   public void delete(String path) throws KeeperException, InterruptedException {
      zookeeper.delete(path,-1);
   }
   /***
    * 删除
    * @param path
    * @return
    * @throws KeeperException
    * @throws InterruptedException
    */
   public void deleteRecursive(String path) throws KeeperException, InterruptedException {
      ZKUtil.deleteRecursive(zookeeper, path);
   }

   public void close() throws InterruptedException {
      zookeeper.close();
   }

   @Override
   public void process(WatchedEvent event) {
      // 连接状态
      Event.KeeperState keeperState = event.getState();
      // 事件类型
      Event.EventType eventType = event.getType();
      // 受影响的path
      String path = event.getPath();
      //step 1:
//        System.out.println("连接状态:"+keeperState+",事件类型："+eventType+",受影响的path:"+path);

      //step:2
      try {
         if(null!=this.exists("/node2",true)) {
            System.out.println("内容:"+ this.getData("/node2", true));
         }
         System.out.println("连接状态:"+keeperState+",事件类型："+eventType+",受影响的path:"+path);
      } catch (Exception e) {
         e.printStackTrace();
      }
      System.out.println("--------------------");
   }
}

package com.huey.watcher;

/**
 * @author huey China.
 * @Description : 事件 Test
 * @Date Created in 2018/11/18 下午3:50
 */

import org.apache.zookeeper.KeeperException;

public class ZookeeperWatcherTest {
    public static void main(String[] args) throws KeeperException, InterruptedException {

        String dir = "/node2";
        ZookeeperWatcher2 zookeeperCrud = new ZookeeperWatcher2();
        //假设该节点已创建
//      zookeeperCrud.delete(dir);//第一个没有事件，
        zookeeperCrud.createPersistent(dir, "node2");
        zookeeperCrud.setData(dir, "neowef");
//      System.out.println("内2容:"+ zookeeperCrud.getData(dir, true));
        Thread.sleep(Integer.MAX_VALUE);//避免主线程结束，接收不到部分其他事件 demo用
    }


}

3.ACL相关

package com.huey.acl;

import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;

/**
 * @author huey China.
 * @Description : ACL
 * @Date Created in 2018/11/18 下午5:01
 */
public class ZookeeperAcl {

    private String connectString = "192.168.59.2:2181,192.168.59.3:2181,192.168.59.4:2181";
    private ZooKeeper zookeeper;
    /**
     * 认证类型
     */
    final static String scheme = "digest";
    final static String auth = "111";//这个就是用户名

    /****
     * 区分下auth和非auth 需要登录
     */
    public ZookeeperAcl(boolean acl) {
        try {
            this.zookeeper = new ZooKeeper(connectString, 5000, null);
            zookeeper.addAuthInfo(scheme, auth.getBytes());//这行代码
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    /***
     * No权限认证的
     * @param
     */
    public ZookeeperAcl() {
        try {
            this.zookeeper = new ZooKeeper(connectString, 5000, null);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    /***
     * 创建持久节点
     * @param path
     * @param data
     * @return
     */
    public String createPersistent(String path, String data) {
        try {
            return zookeeper.create(path, data.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        } catch (KeeperException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return null;
    }

    /***
     * 创建持久节点
     * @param path
     * @param data
     * @return
     */
    public String createPersistentAcl(String path, String data) {
        try {
            return zookeeper.create(path, data.getBytes(), ZooDefs.Ids.CREATOR_ALL_ACL, CreateMode.PERSISTENT);
        } catch (KeeperException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return null;
    }


    /***
     * 创建临时节点
     * @param path
     * @param data
     * @return
     */
    public String createEphemeral(String path, String data) {
        try {
            return zookeeper.create(path, data.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
        } catch (KeeperException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return null;
    }

    /***
     * 更新信息
     * @param path
     * @return
     * @throws KeeperException
     * @throws InterruptedException
     */
    public String getData(String path) throws KeeperException, InterruptedException {
        byte data[] = zookeeper.getData(path, false, null);
        data = (data == null) ? "null".getBytes() : data;
        return new String(data);
    }


    /***
     * 更新信息
     * @param path
     * @param data
     * @return
     * @throws KeeperException
     * @throws InterruptedException
     */
    public Stat setData(String path, String data) throws KeeperException, InterruptedException {
        return zookeeper.setData(path, data.getBytes(), -1);
    }

    /***
     * 是否存在
     * @param path
     * @return
     * @throws KeeperException
     * @throws InterruptedException
     */
    public Stat exists(String path) throws KeeperException, InterruptedException {
        return zookeeper.exists(path, false);

    }


    /***
     * 删除
     * @param path
     * @return
     * @throws KeeperException
     * @throws InterruptedException
     */
    public void delete(String path) throws KeeperException, InterruptedException {
        zookeeper.delete(path, -1);
    }

    /***
     * 删除
     * @param path
     * @return
     * @throws KeeperException
     * @throws InterruptedException
     */
    public void deleteRecursive(String path) throws KeeperException, InterruptedException {
        ZKUtil.deleteRecursive(zookeeper, path);
    }

    public void close() throws InterruptedException {
        zookeeper.close();
    }


}

package com.huey.acl;

/**
* @author huey China.
* @Description : ACL Test
* @Date Created in 2018/11/18 下午5:00
*/

import org.apache.zookeeper.KeeperException;


public class ZookeeperAclTest {

   public static void main(String[] args) throws KeeperException, InterruptedException {
      /***这个是没有权限**/
      ZookeeperAcl zookeeperAcl = new ZookeeperAcl();
      //假设该节点不存在
      zookeeperAcl.createPersistent("/hueyNoAcl","2018");
      String data = zookeeperAcl.getData("/hueyNoAcl");
      System.out.println(data);

      /***这个是权限**/
      ZookeeperAcl zookeeperAcl2 = new ZookeeperAcl(true);
      zookeeperAcl2.createPersistentAcl("/hueyAcl","20188");
      data = zookeeperAcl2.getData("/hueyAcl");
      System.out.println(data); // addauth digest 111



   }

}

总结

watcher原理加znode控制，分布式一致性显而易见！待续

.参考

官网：http://zookeeper.apache.org
书籍：从Paxos到Zookeeper
网课： 推荐慕课网图灵学院谷粒学院

《分布式_Zookeeper》_Zookeeper核心概念以及原

官方定义

核心架构图

zoo.cfg配置

涉及角色

特性

Znode（`org.apache.zookeeper.data.Stat implements Record` 数据结构参考）

Watcher (org.apache.zookeeper.watcher 数据结构参考)

ACL（`org.apache.zookeeper.ZooDefs` 数据结构参考）

其他

Java 原生API(`假设zk集群环境以及搭好并有效启动`)

总结

.参考

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

《分布式_Zookeeper》_Zookeeper核心概念以及原

官方定义

核心架构图

zoo.cfg配置

涉及角色

特性

Znode（org.apache.zookeeper.data.Stat implements Record 数据结构参考）

Watcher (org.apache.zookeeper.watcher 数据结构参考)

ACL（org.apache.zookeeper.ZooDefs 数据结构参考）

其他

Java 原生API(假设zk集群环境以及搭好并有效启动)

总结

.参考

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

Znode（`org.apache.zookeeper.data.Stat implements Record` 数据结构参考）

ACL（`org.apache.zookeeper.ZooDefs` 数据结构参考）

Java 原生API(`假设zk集群环境以及搭好并有效启动`)