美文网首页
RestTemplate Httpclient连接池

RestTemplate Httpclient连接池

作者: 卓三阳 | 来源:发表于2020-07-19 19:47 被阅读0次

    RestTemplate 初解


    1.HTTP长连接、短连接

    HTTP协议的长连接和短连接,实质上是TCP协议的长连接和短连接。

    在HTTP/1.0中默认使用短连接。也就是说,客户端和服务器每进行一次HTTP操作,就建立一次连接,任务结束就中断连接。

    从HTTP/1.1起,默认使用长连接,用以保持连接特性。使用长连接的HTTP协议,会在响应头加入这行代码:Connection:keep-alive.在使用长连接的情况下,客户端和服务器之间用于传输HTTP数据的TCP连接不会关闭,客户端再次访问这个服务器时,会继续使用这一条已经建立的连接。Keep-Alive不会永久保持连接,它有一个保持时间,可以在不同的服务器软件(如Apache)中设定这个时间。实现长连接需要客户端和服务端都支持长连接。

    http_connect.png

    HttpClien中可以使用连接池来管理持有连接,同一条TCP链路上,连接是可以复用的。


    2.RestTemplate的几种实现

    RestTemplate有三个实现

    RestTemplate()
    RestTemplate(List<HttpMessageConverter<?>> messageConverters)
    RestTemplate(ClientHttpRequestFactory requestFactory)。

    下面主要介绍RestTemplate(ClientHttpRequestFactory requestFactory)


    image.png

    现在来看下其它几种客户端的引入。在ClientHttpRequestFactory的实现那张图中列出了RestTemplate的几种REST Client的封装。其中最常用的有以下三种:

    SimpleClientHttpRequestFactory(封装URLConnection) //默认
    HttpComponentsClientHttpRequestFactory(封装HttpClient)
    OkHttp3ClientHttpRequestFactory(封装OKHttp)

    注意:RestTemplate默认的ClientHttpRequestFactor为SimpleClientHttpRequestFactory,默认的超时时间设置为-1,若出现服务器宕机的情况,该连接将永远不会被释放。

    @Configuration
    public class RestConfig {
    
        @Bean
        public RestTemplate restTemplate(){
            RestTemplate restTemplate = new RestTemplate();
            return restTemplate;
        }
    
        @Bean("urlConnectionTemplate")
        public RestTemplate urlConnectionRestTemplate(){
            RestTemplate restTemplate = new RestTemplate(new SimpleClientHttpRequestFactory());
            return restTemplate;
        }
    
        @Bean("httpClientTemplate")
        public RestTemplate httpClientRestTemplate(){
            RestTemplate restTemplate = new RestTemplate(new HttpComponentsClientHttpRequestFactory());
            return restTemplate;
        }
    
        @Bean("OKHttp3Template")
        public RestTemplate OKHttp3RestTemplate(){
            RestTemplate restTemplate = new RestTemplate(new OkHttp3ClientHttpRequestFactory());
            return restTemplate;
        }
    }
    
    

    3.RestTemplate Httpclient连接池

    @Configuration
    @Slf4j
    public class HttpClientConfig {
    
      @Autowired
      private HttpClientPoolConfig httpClientPoolConfig;
    
    /**
    * 原生的restTemplate
    */
      @Bean
      public RestTemplate restTemplate() {
           return new RestTemplate();
      }
    
    
     /**
       * httpClientTemplate
       */
      @Bean(name = "httpClientTemplate")
      public RestTemplate restTemplate(ClientHttpRequestFactory factory) {
           return createRestTemplate(factory);
      }
    
    
      private RestTemplate createRestTemplate(ClientHttpRequestFactory factory) {
          //1.实例化使用httpclient的RestTemplate
           RestTemplate restTemplate = new RestTemplate(factory);
          //2.采用RestTemplate内部的MessageConverter
          //重新设置StringHttpMessageConverter字符集,解决中文乱码问题
            modifyDefaultCharset(restTemplate);
          //3.设置错误处理器
           restTemplate.setErrorHandler(new DefaultResponseErrorHandler());
           return restTemplate;
      }
    
    
    
      /**
       * 创建HTTP客户端工厂
       */
      @Bean(name = "clientHttpRequestFactory")
      public ClientHttpRequestFactory clientHttpRequestFactory() {
     
          //1.检查配置maxTotalConnection 和 maxConnectionPerRoute
         if (httpClientPoolConfig.getMaxTotalConnect() <= 0) {
            throw new IllegalArgumentException("invalid maxTotalConnection: " + 
                 httpClientPoolConfig.getMaxTotalConnect());
          }
         if (httpClientPoolConfig.getMaxConnectPerRoute() <= 0) {
          throw new IllegalArgumentException("invalid maxConnectionPerRoute: " + 
                httpClientPoolConfig.getMaxConnectPerRoute());
        }
         //2.创建clientHttpRequestFactory
        HttpComponentsClientHttpRequestFactory clientHttpRequestFactory = new 
             HttpComponentsClientHttpRequestFactory(httpClient());
        // 连接超时
        clientHttpRequestFactory.setConnectTimeout(httpClientPoolConfig.getConnectTimeout());
        // 数据读取超时时间,即SocketTimeout
        clientHttpRequestFactory.setReadTimeout(httpClientPoolConfig.getReadTimeout());
        // 从连接池获取请求连接的超时时间,不宜过长,必须设置,比如连接不够用时,时间过长将是灾难性的
     clientHttpRequestFactory.setConnectionRequestTimeout(httpClientPoolConfig.getConnectionRequestTimout());
        return clientHttpRequestFactory;
      }
    
    
    
      /**
       * 配置httpClient
       *
       * @return
       */
      @Bean
      public HttpClient httpClient() {
        HttpClientBuilder httpClientBuilder = HttpClientBuilder.create();
        try {
          //设置信任ssl访问
          SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(null, (arg0, arg1) -> true).build();
          httpClientBuilder.setSSLContext(sslContext);
          HostnameVerifier hostnameVerifier = NoopHostnameVerifier.INSTANCE;
          SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(sslContext, hostnameVerifier);
          Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
                  // 注册http和https请求
                  .register("http", PlainConnectionSocketFactory.getSocketFactory())
                  .register("https", sslConnectionSocketFactory).build();
          //使用Httpclient连接池的方式配置(推荐),同时支持netty,okHttp以及其他http框架
          PoolingHttpClientConnectionManager poolingHttpClientConnectionManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);
          // 最大连接数
          poolingHttpClientConnectionManager.setMaxTotal(httpClientPoolConfig.getMaxTotalConnect());
          // 同路由并发数
    poolingHttpClientConnectionManager.setDefaultMaxPerRoute(httpClientPoolConfig.getMaxConnectPerRoute());
          //配置连接池
          httpClientBuilder.setConnectionManager(poolingHttpClientConnectionManager);
          // 重试次数
          httpClientBuilder.setRetryHandler(new DefaultHttpRequestRetryHandler(httpClientPoolConfig.getRetryTimes(), true));
    
          //设置默认请求头
          List<Header> headers = getDefaultHeaders();
          httpClientBuilder.setDefaultHeaders(headers);
          //设置长连接保持策略
          httpClientBuilder.setKeepAliveStrategy(connectionKeepAliveStrategy());
           /*启动清理线程,也可以启用用CloseableHttpClient的清理线程
            Thread t = new IdleConnectionMonitorThread(connectionManager);
            t.setName("httpconnections-pool-evict-thread"); 
            t.start();
           */
          //设置后台线程剔除失效连接
          httpClientBuilder.evictExpiredConnections();
          httpClientBuilder.victIdleConnections(httpClientPoolConfig.getMaxIdleTime(), TimeUnit.MILLISECONDS)
          return httpClientBuilder.build();
        } catch (KeyManagementException | NoSuchAlgorithmException | KeyStoreException e) {
          log.error("初始化HTTP连接池出错", e);
        }
        return null;
      }
    
    
      /**
       * 配置长连接保持策略
       * @return
       */
      public ConnectionKeepAliveStrategy connectionKeepAliveStrategy(){
        return (response, context) -> {
          // Honor 'keep-alive' header
          HeaderElementIterator it = new BasicHeaderElementIterator(
                  response.headerIterator(HTTP.CONN_KEEP_ALIVE));
          while (it.hasNext()) {
            HeaderElement he = it.nextElement();
            log.info("HeaderElement:{}", JSON.toJSONString(he));
            String param = he.getName();
            String value = he.getValue();
            if (value != null && "timeout".equalsIgnoreCase(param)) {
              try {
                return Long.parseLong(value) * 1000;
              } catch(NumberFormatException ignore) {
                log.error("解析长连接过期时间异常",ignore);
              }
            }
          }
          HttpHost target = (HttpHost) context.getAttribute(
                  HttpClientContext.HTTP_TARGET_HOST);
          //如果请求目标地址,单独配置了长连接保持时间,使用该配置
          Optional<Map.Entry<String, Integer>> any = Optional.ofNullable(httpClientPoolProperties.getKeepAliveTargetHost()).orElseGet(HashMap::new)
                  .entrySet().stream().filter(
                  e -> e.getKey().equalsIgnoreCase(target.getHostName())).findAny();
          //否则使用默认长连接保持时间
          return any.map(en -> en.getValue() * 1000L).orElse(httpClientPoolProperties.getKeepAliveTime() * 1000L);
        };
      }
    
      /**
       * 设置请求头
       *
       * @return
       */
      private List<Header> getDefaultHeaders() {
        List<Header> headers = new ArrayList<>();
        headers.add(new BasicHeader("User-Agent",
                "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.16 Safari/537.36"));
        headers.add(new BasicHeader("Accept-Encoding", "gzip,deflate"));
        headers.add(new BasicHeader("Accept-Language", "zh-CN"));
        headers.add(new BasicHeader("Connection", "Keep-Alive"));
        return headers;
      }
    
      /**
       * 修改默认的字符集类型为utf-8
       *
       * @param restTemplate
       */
      private void modifyDefaultCharset(RestTemplate restTemplate) {
        List<HttpMessageConverter<?>> converterList = restTemplate.getMessageConverters();
        HttpMessageConverter<?> converterTarget = null;
        for (HttpMessageConverter<?> item : converterList) {
          if (StringHttpMessageConverter.class == item.getClass()) {
            converterTarget = item;
            break;
          }
        }
        if (null != converterTarget) {
          converterList.remove(converterTarget);
        }
        Charset defaultCharset = Charset.forName(httpClientPoolConfig.getCharset());
        converterList.add(1, new StringHttpMessageConverter(defaultCharset));
      }
    }
    
        // pool监测线程,定期移除过期连接、空闲连接
    /*
        public static class IdleConnectionMonitorThread extends Thread {
    
            private final HttpClientConnectionManager connMgr;
            public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
                super();
                this.connMgr = connMgr;
            }
    
            @Override
            public void run() {
                try {
                    while (true) {
                        synchronized (this) {
                            Thread.sleep(5000);
                            connMgr.closeExpiredConnections();
                            connMgr.closeIdleConnections(5, TimeUnit.SECONDS);
                        }
                    }
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
    */
    
    
    

    使用restTemplate,则无需关闭响应的连接。RestTemplate会为您执行此操作,以验证相同的内容,了解RestTemplate.java

    注意点

    1.对于整个应用,一个HttpClient就够了,HttpClient就像一个小型的浏览器。
    2.启用后台监控线程检查连接
    HttpClient4.4版本之前,在从连接池中获取重用连接的时候会检查下是否过期,过期则清理。之后的版本则不同,会有一个单独的线程来扫描连接池中的连接,发现有离最近一次使用超过设置的时间后,就会清理。只有在HttpClientBuilder手动设置后,才会开启清理过期与空闲连接。
    3.HTTP请求时connectionRequestTimeout 、connectionTimeout、socketTimeout三个超时时间的含义
    (1)connectionRequestTimout
    从连接池获取连接的timeout
    (2)connetionTimeout
    客户端和服务器建立连接的timeout
    就是http请求的三个阶段,一:建立连接;二:数据传送;三,断开连接。超时后ConnectionTimeOutException
    (3)socketTimeout即ReadTimeout
    指客户端和服务器建立连接后,客户端从服务器读取数据的timeout,超出后会抛出SocketTimeOutException


    参考

    SpringBoot的restTemplate整合HttpClient连接池及配置
    SpringBoot系列 - 使用RestTemplate

    相关文章

      网友评论

          本文标题:RestTemplate Httpclient连接池

          本文链接:https://www.haomeiwen.com/subject/mdskcktx.html