Memcached一致性哈希测试

1. 准备

• 两台已经装好memached的机器,假如ip分别为192.168.1.2和192.168.1.3
• spy memcached 客户端jar包
• 启动memached

2. 编写Java测试程序

• 测试哈希一致性的写入

    @Test
    public void testSetWithConsistentHashing() throws IOException, InterruptedException {
        ConnectionFactory connFactory = new ConnectionFactoryBuilder().setLocatorType(Locator.CONSISTENT).build();
        MemcachedClient clusterClient = new MemcachedClient(
                                                            connFactory,
                                                            AddrUtil.getAddresses("192.168.6.201:11211 192.168.6.205:11211"));

        MemcachedClient clientOne = new MemcachedClient(AddrUtil.getAddresses("192.168.6.201:11211"));
        MemcachedClient clientTwo = new MemcachedClient(AddrUtil.getAddresses("192.168.6.205:11211"));

        // 放1000个
        for (int i = 0; i < 1000; i++) {
            clusterClient.set("test" + i, 300, "test" + i);
        }

        while (true) {
            System.out.println("=====================  in one  =================================");
            // 从one中获取,看是否存在,存在的key
            int oneCount = 0;
            for (int i = 0; i < 1000; i++) {
                Object value = clientOne.get("test" + i);
                if (value != null) {
                    oneCount++;
                    System.out.println(value);
                }
            }

            Assert.assertTrue(oneCount > 0);

            Thread.sleep(3 * 1000);

            System.out.println("======================= in two   ===============================");

            // 从one中获取,看是否存在,存在的key
            int twoCount = 0;
            for (int i = 0; i < 1000; i++) {
                Object value = clientTwo.get("test" + i);
                if (value != null) {
                    twoCount++;
                    System.out.println(value);
                }
            }
            Assert.assertTrue(twoCount > 0);
        }
    }

• 改造程序一,测试写入和读取过程中启动或者停掉加入的节点机器

    @Test
    public void testGetValueWithConsistentHashing() throws IOException, InterruptedException {
        ConnectionFactory connFactory = new ConnectionFactoryBuilder().setLocatorType(Locator.CONSISTENT).build();
        MemcachedClient clusterClient = new MemcachedClient(
                                                            connFactory,
                                                            AddrUtil.getAddresses("192.168.6.201:11211 192.168.6.205:11211"));
        MemcachedClient clientOne = new MemcachedClient(AddrUtil.getAddresses("192.168.6.201:11211"));
        MemcachedClient clientTwo = new MemcachedClient(AddrUtil.getAddresses("192.168.6.205:11211"));
        while (true) {
            try {

                // 存数据
                for (int i = 0; i < 1000; i++) {
                    clusterClient.set("test" + i, 300, "test" + i);
                }

                // 校验one中是否有数据
                System.out.println("=====================  in one  =================================");
                int oneCount = 0;
                for (int i = 0; i < 1000; i++) {
                    Object value = clientOne.get("test" + i);
                    if (value != null) {
                        oneCount++;
                        if (oneCount > 3) {
                            continue;
                        }
                        System.out.println(value);
                    }
                }
                System.out.println("one count: " + oneCount);

                // //校验two中是否有数据
                System.out.println("=====================  in two  =================================");
                int twoCount = 0;
                for (int i = 0; i < 1000; i++) {
                    Object value = clientTwo.get("test" + i);
                    if (value != null) {
                        twoCount++;
                        if (twoCount > 3) {
                            continue;
                        }
                        System.out.println(value);
                    }
                }
                System.out.println("two count: " + twoCount);

                Thread.sleep(3 * 1000);
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }

3. 测试

• 同时开启两台memcached,然后执行第一个测试程序,结果执行通过,发现两台机器的memcached中都有要设置的数据,而且是不同的.

• 同时开启两台memcached,然后执行第二个测试程序,开始执行征程.然后停掉其中一台,没多长时间,测试程序抛出异常

2014-08-29 19:27:57.752 INFO net.spy.memcached.MemcachedConnection:  Reconnecting {QA sa=/192.168.1.2:11211, #Rops=0, #Wops=23, #iq=0, topRop=null, topWop=net.spy.memcached.protocol.ascii.StoreOperationImpl@567a380b, toWrite=0, interested=0}
2014-08-29 19:27:57.753 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@5e94fd79
2014-08-29 19:27:57.754 INFO net.spy.memcached.MemcachedConnection:  Reconnecting due to failure to connect to {QA sa=/192.168.1.2:11211, #Rops=0, #Wops=23, #iq=0, topRop=null, topWop=net.spy.memcached.protocol.ascii.StoreOperationImpl@567a380b, toWrite=0, interested=0}
java.net.ConnectException: 拒绝连接
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
    at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:313)
    at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:199)
    at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1622)
2014-08-29 19:27:57.754 WARN net.spy.memcached.MemcachedConnection:  Closing, and reopening {QA sa=/192.168.1.2:11211, #Rops=0, #Wops=23, #iq=0, topRop=null, topWop=net.spy.memcached.protocol.ascii.StoreOperationImpl@567a380b, toWrite=0, interested=0}, attempt 1.

然后就隔一段时间抛出一次异常,且每次抛出异常的时间是不断推迟的,具体推迟到多久这个尚未测试.同时所要写入的1000个值全部写入到了另一台机器中.

• 开启停掉的那台机器,客户端自动重新链接了,同时恢复了对刚刚恢复的机器的写入和读取

2014-08-29 20:03:06.157 WARN net.spy.memcached.MemcachedConnection:  Could not redistribute to another node, retrying primary node for test0.
2014-08-29 20:03:06.234 INFO net.spy.memcached.MemcachedConnection:  Reconnecting {QA sa=/192.168.1.2:11211, #Rops=0, #Wops=22, #iq=0, topRop=null, topWop=net.spy.memcached.protocol.ascii.StoreOperationImpl@6d2cb32a, toWrite=0, interested=0}
2014-08-29 20:03:06.235 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@51906e2
2014-08-29 20:03:06.476 INFO net.spy.memcached.MemcachedConnection:  Reconnecting {QA sa=/192.168.1.2:11211, #Rops=0, #Wops=3, #iq=0, topRop=null, topWop=net.spy.memcached.protocol.ascii.GetOperationImpl@59b3d57, toWrite=0, interested=0}
2014-08-29 20:03:06.477 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@399931f9

4. 结果

spymemcached 客户端对memcached哈希一致性比较i理想,在有多台memcached的情况下,挂掉一台,不会造成全部损失,但会失去这台机器的缓存,导致这部分数据可能会落在数据库上,相比单节点挂掉,情况要理想的多.

5. 探索

如果支持主备方案,那么情况会更加理想.