美文网首页
Spring Cloud Hystrix ThreadPool的

Spring Cloud Hystrix ThreadPool的

作者: 干货满满张哈希 | 来源:发表于2019-11-28 19:38 被阅读0次

    BUG背景

    JDK: 11.0.4
    Spring Cloud Finchley.SR3

    相关配置:

    #开启hystrix
    feign.hystrix.enabled=true
    #关闭断路器
    hystrix.command.default.circuitBreaker.enabled=false
    #禁用hystrix远程调用超时时间
    hystrix.command.default.execution.timeout.enabled=false
    hystrix.threadpool.default.coreSize=50
    

    Hystrix隔离策略:线程隔离

    在Feign调用的时候,会报错:

    Caused by: java.util.concurrent.ExecutionException: Observable onError
            at rx.internal.operators.BlockingOperatorToFuture$2.getValue(BlockingOperatorToFuture.java:118) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.BlockingOperatorToFuture$2.get(BlockingOperatorToFuture.java:102) ~[rxjava-1.3.8.jar!/:1.3.8]
            at com.netflix.hystrix.HystrixCommand$4.get(HystrixCommand.java:423) ~[hystrix-core-1.5.18.jar!/:1.5.18]
            at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:344) ~[hystrix-core-1.5.18.jar!/:1.5.18]
            ... 30 more
    Caused by: java.lang.IllegalArgumentException
            at java.util.concurrent.ThreadPoolExecutor.setCorePoolSize(ThreadPoolExecutor.java:1535) ~[?:?]
            at com.netflix.hystrix.HystrixThreadPool$HystrixThreadPoolDefault.touchConfig(HystrixThreadPool.java:230) ~[hystrix-core-1.5.18.jar!/:1.5.18]
            at com.netflix.hystrix.HystrixThreadPool$HystrixThreadPoolDefault.getScheduler(HystrixThreadPool.java:205) ~[hystrix-core-1.5.18.jar!/:1.5.18]
            at com.netflix.hystrix.AbstractCommand.executeCommandWithSpecifiedIsolation(AbstractCommand.java:710) ~[hystrix-core-1.5.18.jar!/:1.5.18]
            at com.netflix.hystrix.AbstractCommand.executeCommandAndObserve(AbstractCommand.java:638) ~[hystrix-core-1.5.18.jar!/:1.5.18]
            at com.netflix.hystrix.AbstractCommand.applyHystrixSemantics(AbstractCommand.java:546) ~[hystrix-core-1.5.18.jar!/:1.5.18]
            at com.netflix.hystrix.AbstractCommand.access$200(AbstractCommand.java:60) ~[hystrix-core-1.5.18.jar!/:1.5.18]
            at com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:419) ~[hystrix-core-1.5.18.jar!/:1.5.18]
            at com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:413) ~[hystrix-core-1.5.18.jar!/:1.5.18]
            at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:46) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.Observable.unsafeSubscribe(Observable.java:10327) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.Observable.unsafeSubscribe(Observable.java:10327) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.Observable.unsafeSubscribe(Observable.java:10327) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.Observable.unsafeSubscribe(Observable.java:10327) ~[rxjava-1.3.8.jar!/:1.3.8]
            at rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:51) ~[rxjava-1.3.8.jar!/:1.3.8]
    
    

    重启有时候复现,有时候不复现,估计是和类加载还有初始化顺序有关的问题。

    问题定位

    查看问题源代码, 对于每个微服务调用,初始化Hytrix线程池,动态读取配置:

    private void touchConfig() {
        final int dynamicCoreSize = properties.coreSize().get();
        final int configuredMaximumSize = properties.maximumSize().get();
        int dynamicMaximumSize = properties.actualMaximumSize();
        final boolean allowSizesToDiverge = properties.getAllowMaximumSizeToDivergeFromCoreSize().get();
        boolean maxTooLow = false;
    
        if (allowSizesToDiverge && configuredMaximumSize < dynamicCoreSize) {
            //if user sets maximum < core (or defaults get us there), we need to maintain invariant of core <= maximum
            dynamicMaximumSize = dynamicCoreSize;
            maxTooLow = true;
        }
    
        // In JDK 6, setCorePoolSize and setMaximumPoolSize will execute a lock operation. Avoid them if the pool size is not changed.
        if (threadPool.getCorePoolSize() != dynamicCoreSize || (allowSizesToDiverge && threadPool.getMaximumPoolSize() != dynamicMaximumSize)) {
            if (maxTooLow) {
                logger.error("Hystrix ThreadPool configuration for : " + metrics.getThreadPoolKey().name() + " is trying to set coreSize = " +
                        dynamicCoreSize + " and maximumSize = " + configuredMaximumSize + ".  Maximum size will be set to " +
                        dynamicMaximumSize + ", the coreSize value, since it must be equal to or greater than the coreSize value");
            }
            threadPool.setCorePoolSize(dynamicCoreSize);
            threadPool.setMaximumPoolSize(dynamicMaximumSize);
        }
    
        threadPool.setKeepAliveTime(properties.keepAliveTimeMinutes().get(), TimeUnit.MINUTES);
    }
    

    报错是在threadPool.setCorePoolSize(dynamicCoreSize);,看下为何:

    public void setCorePoolSize(int corePoolSize) {
            if (corePoolSize < 0 || maximumPoolSize < corePoolSize)
                throw new IllegalArgumentException();
        //忽略其他代码
    }
    

    发现还有maximumPoolSize < corePoolSize的判断,所以,需要先设置MaximumPoolSize,之后设置CorePoolSize,这样可以避免这个问题

    相关ISSUES与解决方案

    看了下社区,已经有ISSUE了:https://github.com/Netflix/Hystrix/issues/1874 还有对应的PULL REQUEST:https://github.com/Netflix/Hystrix/pull/1877

    但是目前,还没有合并,只好自己改了,项目中增加同名同路径替换类,修改:

    //jdk9以上的版本,设置coreSize会验证coreSize必须小于maxSize,所以先改变max再设置core
                    threadPool.setMaximumPoolSize(dynamicMaximumSize);
                    threadPool.setCorePoolSize(dynamicCoreSize);
    
    

    相关文章

      网友评论

          本文标题:Spring Cloud Hystrix ThreadPool的

          本文链接:https://www.haomeiwen.com/subject/ojhiwctx.html