问题代码
Pipeline pip = jedis.pipelined();
for (String jpushId : allUsers) {
String key = prefix + jpushId + "_" + urlMd5;
pip.set(key,"1");
pip.expire(key,60*60*24*7);
if(++count%5000==0){
pip.sync();
}
}
pip.sync();
现象描述
数据会莫名其妙的丢失,没有存储到redis中,而且程序也不会报错
参考https://stackoverflow.com/questions/26780531/data-loss-in-redis-through-jedis
原因:jedis 超时或者网络问题
解决办法,每批次关闭pipline redis,获取新的redis实例和管道
Pipeline pip = jedis.pipelined();
for (String jpushId : allUsers) {
String key = prefix + jpushId + "_" + urlMd5;
pip.set(key,"1");
pip.expire(key,60*60*24*7);
if(++count%5000==0){
pip.sync();
jedis.close();
jedis = jedisClient.getCacheClient();
pip = jedis.pipelined();
}
}
pip.sync();
验证 exists
问题 public Long exists(final String... keys)
这个参数批量传入,怎么办?
解决办法
String[] arr = list.toArray(new String[list.size()])
public void validationRedisCnt(List<String> users,String md5){
List<String> list = new ArrayList<>();
Jedis jedis = jedisClient.getCacheClient();
int count = 0;
for (String jpushId : users) {
String key = prefix + jpushId + "_" + md5;
list.add(key);
if(++count%5000==0){
jedis.close();
jedis = jedisClient.getCacheClient();
String[] arr = list.toArray(new String[list.size()]);
long cnt = jedis.exists(arr);
check(md5, list, jedis, cnt);
list = new ArrayList<>();
}
}
String[] arr = list.toArray(new String[list.size()]);
long cnt = jedis.exists(arr);
check(md5, list, jedis, cnt);
jedis.close();
}
public void check(String md5, List<String> list, Jedis jedis, long cnt) {
if(list.size()!=cnt){
logger.error("数据缺失 {}",String.join(",",list));
//补充
Pipeline pip = jedis.pipelined();
for (String s : list) {
String sk = prefix + s + "_" + md5;
pip.set(sk,"1");
pip.expire(sk,60*60*24*7);
}
pip.sync();
}
}
网友评论