美文网首页
redis pipline 丢失数据

redis pipline 丢失数据

作者: 邵红晓 | 来源:发表于2020-07-01 16:20 被阅读0次

问题代码

        Pipeline pip = jedis.pipelined();
        for (String jpushId : allUsers) {
            String key = prefix + jpushId + "_" + urlMd5;
            pip.set(key,"1");
            pip.expire(key,60*60*24*7);
            if(++count%5000==0){
                pip.sync();
            }
        }
        pip.sync();

现象描述

数据会莫名其妙的丢失,没有存储到redis中,而且程序也不会报错
参考https://stackoverflow.com/questions/26780531/data-loss-in-redis-through-jedis
原因:jedis 超时或者网络问题

解决办法,每批次关闭pipline redis,获取新的redis实例和管道

 Pipeline pip = jedis.pipelined();
        for (String jpushId : allUsers) {
            String key = prefix + jpushId + "_" + urlMd5;
            pip.set(key,"1");
            pip.expire(key,60*60*24*7);
            if(++count%5000==0){
                pip.sync();
                jedis.close();
                jedis = jedisClient.getCacheClient();
                pip = jedis.pipelined();
            }
        }
        pip.sync();

验证 exists

问题 public Long exists(final String... keys)这个参数批量传入,怎么办?
解决办法
String[] arr = list.toArray(new String[list.size()])

 public void validationRedisCnt(List<String> users,String md5){
        List<String> list = new ArrayList<>();
        Jedis jedis = jedisClient.getCacheClient();
        int count = 0;
        for (String jpushId : users) {
            String key = prefix + jpushId + "_" + md5;
            list.add(key);
            if(++count%5000==0){
                jedis.close();
                jedis = jedisClient.getCacheClient();
                String[] arr = list.toArray(new String[list.size()]);
                long cnt = jedis.exists(arr);
                check(md5, list, jedis, cnt);
                list = new ArrayList<>();
            }
        }
        String[] arr = list.toArray(new String[list.size()]);
        long cnt = jedis.exists(arr);
        check(md5, list, jedis, cnt);
        jedis.close();
    }

    public void check(String md5, List<String> list, Jedis jedis, long cnt) {
        if(list.size()!=cnt){
            logger.error("数据缺失 {}",String.join(",",list));
            //补充
            Pipeline pip = jedis.pipelined();
            for (String s : list) {
                String sk =  prefix + s + "_" + md5;
                pip.set(sk,"1");
                pip.expire(sk,60*60*24*7);
            }
            pip.sync();
        }
    }

相关文章

网友评论

      本文标题:redis pipline 丢失数据

      本文链接:https://www.haomeiwen.com/subject/cychqktx.html