背景
本司渠道服务上线后每运行一周左右,内存呈缓慢上升趋势,并最终引起OOM
问题描述
线上渠道服务,是负责整个对外的渠道接入,有段时间突然发现服务会自动宕机,经过重启又稳定运行,但慢慢经过一周左右的时间又自动宕机,后经过grafana监控到系统指标中Memory useage指标趋势是呈缓慢上升趋势,最终造成系统OOM
问题排查过程
线上通过jmap 抓到dump文件,经过排查发现 加解密提供者 BouncyCastleProvider 这个对象占用了大量内存并未释放的趋势,后逐渐通过代码排查发现
BouncyCastleProvider 的生成 是因为调用了Cipher.getInstance 方法
关键代码片段
/**
* 3DES加密
*
* @param value 普通文本
* @param secretKey
* @return
* @throws Exception
*/
public static String encrypt3DES(String value, String secretKey) throws Exception {
String plainText = null;
byte[] keyBytes = newByte(24);
to24Key(secretKey, keyBytes);
KeySpec dks = new DESedeKeySpec(keyBytes);
SecretKey secKey = SecretKeyFactory.getInstance(Algorithm).generateSecret(dks);
Cipher cipher = Cipher.getInstance("DESede/ECB/NoPadding");
cipher.init(Cipher.ENCRYPT_MODE, secKey);
byte[] srcBytes = value.getBytes(encoding);
int srcLen = srcBytes.length;
int valuelen = srcLen + 1;
int encLen = ((valuelen % 8) == 0) ? valuelen : ((valuelen / 8 + 1) * 8);
byte[] encBytes = newByte(encLen);
encBytes[0] = (byte) srcLen;
System.arraycopy(srcBytes, 0, encBytes, 1, srcLen);
// 正式执行解密操作
byte[] encryptBytes = cipher.doFinal(encBytes);
plainText = Base64.encode(encryptBytes);
return plainText;
}
由上代码可知,加解密的主要对象Cipher是这么实例化的
Cipher cipher = Cipher.getInstance("DESede/ECB/NoPadding");
继续跟进getInstance方法内部
public static final Cipher getInstance(String var0) throws NoSuchAlgorithmException, NoSuchPaddingException {
List var1 = getTransforms(var0);
ArrayList var2 = new ArrayList(var1.size());
Iterator var3 = var1.iterator();
while(var3.hasNext()) {
Cipher.Transform var4 = (Cipher.Transform)var3.next();
var2.add(new ServiceId("Cipher", var4.transform));
}
List var11 = GetInstance.getServices(var2);
Iterator var12 = var11.iterator();
Exception var5 = null;
while(true) {
Service var6;
Cipher.Transform var7;
int var8;
do {
do {
do {
if (!var12.hasNext()) {
throw new NoSuchAlgorithmException("Cannot find any provider supporting " + var0, var5);
}
var6 = (Service)var12.next();
} while(!JceSecurity.canUseProvider(var6.getProvider()));
var7 = getTransform(var6, var1);
} while(var7 == null);
var8 = var7.supportsModePadding(var6);
} while(var8 == 0);
if (var8 == 2) {
return new Cipher((CipherSpi)null, var6, var12, var0, var1);
}
try {
CipherSpi var9 = (CipherSpi)var6.newInstance((Object)null);
var7.setModePadding(var9);
return new Cipher(var9, var6, var12, var0, var1);
} catch (Exception var10) {
var5 = var10;
}
}
}
由上代码可发现 最终会走到 JceSecurity.canUseProvider(var6.getProvider()) 方法中,此var6.getProvider() 是每次调用该getInstance方法的时候,每次都会生成的,根据此线索继续往下跟进canUseProvider方法内部
static synchronized Exception getVerificationResult(Provider var0) {
Object var1 = verificationResults.get(var0);
if (var1 == PROVIDER_VERIFIED) {
return null;
} else if (var1 != null) {
return (Exception)var1;
} else if (verifyingProviders.get(var0) != null) {
return new NoSuchProviderException("Recursion during verification");
} else {
Exception var3;
try {
verifyingProviders.put(var0, Boolean.FALSE);
URL var2 = getCodeBase(var0.getClass());
verifyProviderJar(var2);
verificationResults.put(var0, PROVIDER_VERIFIED);
var3 = null;
return var3;
} catch (Exception var7) {
verificationResults.put(var0, var7);
var3 = var7;
} finally {
verifyingProviders.remove(var0);
}
return var3;
}
}
此段代码的核心是 verifyingProviders.put(var0, Boolean.FALSE); var0,是外部传进来的var6.getProvider(),之前分析过var6.getProvider()是每次调用都会重新生成,那么这里每次会将重新生成的Provider插入到verifyingProviders中;(马上要破案了)
private static final Map<Provider, Object> verifyingProviders = new IdentityHashMap();
我们可以看到verifyingProviders是一个静态的IdentityHashMap,这个Map在存储类的时候并不是使用类的equals方法来判断是否Key已经存在,而是使用 == 来判断是否Key已经存在的;换句话说就是当两个对象不 == 那么此Map就会将这个对象存进去,由于静态对象的关系会造成系统内存不会释放,从而导致程序OOM;
解决方案
通过查看其源码,发现 Cipher.getInstance 方法有一个重载方法,Cipher getInstance(String var0, Provider var1),此方法由外部传入一个Provider,这样外部只要保证此Provider是全局唯一即可;
后续改造方案
static Provider provider = new BouncyCastleProvider();
/**
* 3DES加密
*
* @param value 普通文本
* @param secretKey
* @return
* @throws Exception
*/
public static String encrypt3DES(String value, String secretKey) throws Exception {
xxxxxx
Cipher cipher = Cipher.getInstance("DESede/ECB/NoPadding", provider);
xxxxxx
}
网友评论