okhttp源码解析(五):代理和DNS

作者: 珠穆朗玛小王子 | 来源:发表于2018-07-31 16:15 被阅读13次

    前言

    之前我们分析了okhttp的重试机制,发现在获取可用地址的时候,都需要遍历一个路由选择器,里面保存了可用的地址,那么这些地址是从哪来的呢?这就是本篇分析的重点。

    首先我们简单理解一下代理和DNS的概念:

    代理:通过另一台服务器或ip,帮助我们进行网络请求的转发,例如创建的抓包工具。

    DNS:万维网上作为域名和IP地址相互映射的一个分布式数据库,能够使用户更方便的访问互联网,而不用去记住能够被机器直接读取的IP数串。常用的有阿里云的DNS服务。

    从他们的概念之间,我们可以知道,使用代理的话网速是变慢的风险,而DNS不仅不会增加网络请求的成本,还会节省访问网络的时间,所以DNS服务已经十分普遍。

    (突然回忆起刚上班时,公司内网封了QQ地址,每隔一段时间就需要更换代理的日子……)

    正文

    首先看看怎么设置代理和DNS:

    OkHttpClient okHttpClient = new OkHttpClient.Builder()
                    // 多个代理
                    .proxySelector(new ProxySelector() {
                        @Override
                        public List<Proxy> select(URI uri) {
                            return null;
                        }
    
                        @Override
                        public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
    
                        }
                    })
                    // 单独的代理
                    .proxy(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("www.baidu.com", 8888)))
                    // dns
                    .dns(new Dns() {
                        @Override
                        public List<InetAddress> lookup(String hostname) {
                            return null;
                        }
                    })
                    .build();
    

    代理和DNS信息都被保存到OkhttpClient对象中:

    proxySelector可以为一个URI设置多个代理,如果地址连接失败还回调connectFailed;

    proxy设置单独的全局代理,他的优先级高于proxySelecttor;

    dns用法和proxySelecttor类似,可以返回多个地址。

    接下来我们看看okhttp到底是怎么使用代理和DNS的,回忆之前的分析,我们发现处理网络连接,释放等操作都是在StreamAllocation中:

    StreamAllocation streamAllocation = new StreamAllocation(client.connectionPool(),
                    createAddress(request.url()), call, eventListener, callStackTrace);
    

    其中把代理和dns信息设置到网络请求中是在createAddress()方法中:

    private Address createAddress(HttpUrl url) {
            SSLSocketFactory sslSocketFactory = null;
            HostnameVerifier hostnameVerifier = null;
            CertificatePinner certificatePinner = null;
            if (url.isHttps()) {
                sslSocketFactory = client.sslSocketFactory();
                hostnameVerifier = client.hostnameVerifier();
                certificatePinner = client.certificatePinner();
            }
    
            return new Address(url.host(), url.port(), client.dns(), client.socketFactory(),
                    sslSocketFactory, hostnameVerifier, certificatePinner, client.proxyAuthenticator(),
                    client.proxy(), client.protocols(), client.connectionSpecs(), client.proxySelector());
        }
    
    

    在Address的构造方法中,我们看到了熟悉的DNS,ProxySelector和Proxy,Address只是封装了所有的可以访问的地址信息,功能还是在StreamAllocation中,之前我们看到了findConnection方法是负责找到可用的连接,现在我们开始一步步的分析他的代码:

    private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout,
          int pingIntervalMillis, boolean connectionRetryEnabled) throws IOException {
        boolean foundPooledConnection = false;
        RealConnection result = null;
        Route selectedRoute = null;
        Connection releasedConnection;
        Socket toClose;
        // 同步访问连接池
        synchronized (connectionPool) {
          if (released) throw new IllegalStateException("released");
          if (codec != null) throw new IllegalStateException("codec != null");
          if (canceled) throw new IOException("Canceled");
    
          // 需要判断是否要释放当前的连接
          releasedConnection = this.connection;
          // 判断是否要释放,在此方法中是设置了,如果要释放this.connection = null
          toClose = releaseIfNoNewStreams();
          // 经过检查以后,发现当前的连接可以继续使用
          if (this.connection != null) {
            // We had an already-allocated connection and it's good.
            result = this.connection;
            releasedConnection = null;
          }
          // 如果不需要继续保留
          if (!reportedAcquired) {
            // If the connection was never reported acquired, don't report it as released!
            releasedConnection = null;
          }
          // 如果没有可以访问的连接
          if (result == null) {
            // Attempt to get a connection from the pool.
            // 从访问池中找到可以访问的地址
            Internal.instance.get(connectionPool, address, this, null);
            // 如果找到了可用的地址
            if (connection != null) {
              foundPooledConnection = true;
              result = connection;
            }
            // 否则使用路由
            else {
              selectedRoute = route;
            }
          }
        }
        // 关闭之前的socket
        closeQuietly(toClose);
    
        if (releasedConnection != null) {
          eventListener.connectionReleased(call, releasedConnection);
        }
        if (foundPooledConnection) {
          eventListener.connectionAcquired(call, result);
        }
        // 找到可以用的地址,直接返回
        if (result != null) {
          // If we found an already-allocated or pooled connection, we're done.
          return result;
        }
    
        ...
        return result;
      }
    

    上面是查找可用连接的第一步,首先判断自己持有的connection是否可用,不可用的话就关闭连接,然后在连接池中找到合适的连接,如果没找到可用的连接,使用路由中的信息。下面是Internal.instance.get(connectionPool, address, this, null)的源码:

    @Nullable RealConnection get(Address address, StreamAllocation streamAllocation, Route route) {
        assert (Thread.holdsLock(this));
        for (RealConnection connection : connections) {
          if (connection.isEligible(address, route)) {
            streamAllocation.acquire(connection, true);
            return connection;
          }
        }
        return null;
      }
    

    代码非常的简单,就是遍历连接池中的connection,看看哪一个能能用使用,如果能够使用,这个设置accquire等于true,因为之后可能要继续使用。

    假设没有可用的连接,接下来就要开始使用路由了,那么我们设置的代理和DNS是在哪里被添加到路由里的呢?

    public StreamAllocation(ConnectionPool connectionPool, Address address, Call call,
          EventListener eventListener, Object callStackTrace) {
        ......
        this.routeSelector = new RouteSelector(address, routeDatabase(), call, eventListener);
        ......
      }
    
    public RouteSelector(Address address, RouteDatabase routeDatabase, Call call,
          EventListener eventListener) {
       ......
        // 准备访问代理
        resetNextProxy(address.url(), address.proxy());
      }
    
    private void resetNextProxy(HttpUrl url, Proxy proxy) {
        // 如果指定了一个代理,那代理保存到proxies中
        if (proxy != null) {
          // If the user specifies a proxy, try that and only that.
          proxies = Collections.singletonList(proxy);
        } else {
          // 从Address中选择这个url的代理
          // Try each of the ProxySelector choices until one connection succeeds.
          List<Proxy> proxiesOrNull = address.proxySelector().select(url.uri());
          proxies = proxiesOrNull != null && !proxiesOrNull.isEmpty()
              ? Util.immutableList(proxiesOrNull)
              : Util.immutableList(Proxy.NO_PROXY);
        }
        // 代理的索引值变为第一个
        nextProxyIndex = 0;
      }
    

    再创建StreamAllocation的时候同时创建了RouteSelector,RouteSelector在构造方法中设置了代理信息,可以看到我们设置的Proxy的优先级大于ProxySelector,两者不会同时使用。

    接下来继续看SteamAllocation在路由中查找可用连接代码:

    // If we need a route selection, make one. This is a blocking operation.
        boolean newRouteSelection = false;
        // 如果有可用的路由,开始遍历
        if (selectedRoute == null && (routeSelection == null || !routeSelection.hasNext())) {
          newRouteSelection = true;
          routeSelection = routeSelector.next();
        }
    
        synchronized (connectionPool) {
          if (canceled) throw new IOException("Canceled");
    
          if (newRouteSelection) {
            // Now that we have a set of IP addresses, make another attempt at getting a connection from
            // the pool. This could match due to connection coalescing.
            // 取出所有的路由信息
            List<Route> routes = routeSelection.getAll();
            for (int i = 0, size = routes.size(); i < size; i++) {
              Route route = routes.get(i);
              // 遍历路由集合,直到找到能用的连接
              Internal.instance.get(connectionPool, address, this, route);
              // 找到连接,跳珠循环
              if (connection != null) {
                foundPooledConnection = true;
                result = connection;
                this.route = route;
                break;
              }
            }
          }
          // 没有找到再遍历下一个路由
          if (!foundPooledConnection) {
            if (selectedRoute == null) {
              // 如果没有找到,再使用下一个路由集合
              selectedRoute = routeSelection.next();
            }
    
            // Create a connection and assign it to this allocation immediately. This makes it possible
            // for an asynchronous cancel() to interrupt the handshake we're about to do.
            route = selectedRoute;
            refusedStreamCount = 0;
            // 找到可用的地址
            result = new RealConnection(connectionPool, selectedRoute);
            // 设置可以直接关闭
            acquire(result, false);
          }
        }
    
        // If we found a pooled connection on the 2nd time around, we're done.
        if (foundPooledConnection) {
          eventListener.connectionAcquired(call, result);
          return result;
        }
    

    这里先看看是否有正在使用的代理集合,如果没有在用的代理集合,这里调用了RouteSelector.next()方法来获得下一个路由集合,为什么这里设置的acquire等于true呢?之前的注释已经说明了,代理不能告诉我们服务器的访问地址信息,所以我们也没有必要保持连接:

    public Selection next() throws IOException {
        if (!hasNext()) {
          throw new NoSuchElementException();
        }
    
        // Compute the next set of routes to attempt.
        List<Route> routes = new ArrayList<>();
        // 循环遍历代理列表
        while (hasNextProxy()) {
          // Postponed routes are always tried last. For example, if we have 2 proxies and all the
          // routes for proxy1 should be postponed, we'll move to proxy2\. Only after we've exhausted
          // all the good routes will we attempt the postponed routes.
          // 得到下一个代理,
          Proxy proxy = nextProxy();
          for (int i = 0, size = inetSocketAddresses.size(); i < size; i++) {
            Route route = new Route(address, proxy, inetSocketAddresses.get(i));
            // 判断是否这个路由最近访问失败过
            if (routeDatabase.shouldPostpone(route)) {
              postponedRoutes.add(route);
            } else {
              // 需要发起访问,添加到列表中
              routes.add(route);
            }
          }
          // 如果已经找到了可以访问的路由,跳出循环
          if (!routes.isEmpty()) {
            break;
          }
        }
        // 如果遍历结束后,可以访问的路由是空的
        if (routes.isEmpty()) {
          // We've exhausted all Proxies so fallback to the postponed routes.
          // 把失败的路由保存到列表中
          routes.addAll(postponedRoutes);
          // 清空之前记录的失败的路由列表
          postponedRoutes.clear();
        }
        // 返回访问的路由集合
        return new Selection(routes);
      }
    

    next()方法最关键的部分是nextProxy()方法,他返回了要添加的地址集合:

    private Proxy nextProxy() throws IOException {
        if (!hasNextProxy()) {
          throw new SocketException("No route to " + address.url().host()
              + "; exhausted proxy configurations: " + proxies);
        }
        Proxy result = proxies.get(nextProxyIndex++);
        // 设置下一个代理集合,这里设置了所有的代理和dns
        resetNextInetSocketAddress(result);
        return result;
      }
    

    这里取出之前的代理Proxy作为参数,传入resetNextInetSocketAddress()方法,这是为什么呢?

    private void resetNextInetSocketAddress(Proxy proxy) throws IOException {
        inetSocketAddresses = new ArrayList<>();
    
        String socketHost;
        int socketPort;
        // 判断代理的类型
        if (proxy.type() == Proxy.Type.DIRECT || proxy.type() == Proxy.Type.SOCKS) {
          socketHost = address.url().host();
          socketPort = address.url().port();
        } else {
          // 得到代理的地址
          SocketAddress proxyAddress = proxy.address();
          if (!(proxyAddress instanceof InetSocketAddress)) {
            throw new IllegalArgumentException(
                "Proxy.address() is not an " + "InetSocketAddress: " + proxyAddress.getClass());
          }
          InetSocketAddress proxySocketAddress = (InetSocketAddress) proxyAddress;
          // 使用代理的host和端口
          socketHost = getHostString(proxySocketAddress);
          socketPort = proxySocketAddress.getPort();
        }
        // 判断端口号是否合法
        if (socketPort < 1 || socketPort > 65535) {
          throw new SocketException("No route to " + socketHost + ":" + socketPort
              + "; port is out of range");
        }
        // 这里是关键,如果代理的类型是Socks,不适用DNS
        if (proxy.type() == Proxy.Type.SOCKS) {
          inetSocketAddresses.add(InetSocketAddress.createUnresolved(socketHost, socketPort));
        } else {
          eventListener.dnsStart(call, socketHost);
    
          // Try each address for best behavior in mixed IPv4/IPv6 environments.
          // dns解析代理地址
          List<InetAddress> addresses = address.dns().lookup(socketHost);
          if (addresses.isEmpty()) {
            throw new UnknownHostException(address.dns() + " returned no addresses for " + socketHost);
          }
    
          eventListener.dnsEnd(call, socketHost, addresses);
          // 把解析的地址添加到列表中
          for (int i = 0, size = addresses.size(); i < size; i++) {
            InetAddress inetAddress = addresses.get(i);
            inetSocketAddresses.add(new InetSocketAddress(inetAddress, socketPort));
          }
        }
      }
    

    首先通过上一个代理,解析出他的host和端口,如果代理是SOCKS类型,不使用DNS,如果不是SOCKS类型,就去获得这个代理的DNS地址列表,继续尝试连接。

    那如果我不设置任何的代理信息,DNS还会执行吗?会的,因为okhttp默认设置的DefaultProxySelector,我们仍然可以在连接失败后,尝试访问url的DNS域名。

    最后是StreamAllocation.findConnection()的最后一步:

    // Do TCP + TLS handshakes. This is a blocking operation.
        result.connect(connectTimeout, readTimeout, writeTimeout, pingIntervalMillis,
            connectionRetryEnabled, call, eventListener);
        routeDatabase().connected(result.route());
    
        Socket socket = null;
        synchronized (connectionPool) {
          reportedAcquired = true;
    
          // Pool the connection.
          Internal.instance.put(connectionPool, result);
    
          // If another multiplexed connection to the same address was created concurrently, then
          // release this connection and acquire that one.
          if (result.isMultiplexed()) {
            socket = Internal.instance.deduplicate(connectionPool, address, this);
            result = connection;
          }
        }
        closeQuietly(socket);
    
        eventListener.connectionAcquired(call, result);
        return result;
    

    第二步中,如果在使用的路由中没有找到可用的连接,会返回RealConnection对象,里面保存了代理的Socks或者DNS信息,调用connect方法建立连接,然后做了去重的判断,防止相同地址保持多个连接。

    到此为止,我们设置的代理和DNS的任务圆满结束。

    总结

    最后我们来总结一下:

    1. Proxy和ProxySelector不可同时使用,同时存在优先使用Proxy。
    2. 如果代理的类型是SOCKS,那么他的DNS不会被使用。
    3. 网络请求会先访问原始域名,失败之后才会使用DNS域名。
    4. 通过代理完成网络操作,不会保持连接,因为我们无法通过代理得到访问的真正地址。

    到这里okhttp源码解析系列暂时告一段落了,我们整体的分析了okhttp的工作过程,然后分别重点分析okhttp的网络读写,缓存机制,重试机制,代理和DNS,之后继续使用okhttp会更加得心应手。

    如果之后还有新发现再继续补充,希望这一系列对大家对okhttp的理解有所帮助。

    相关文章

      网友评论

        本文标题:okhttp源码解析(五):代理和DNS

        本文链接:https://www.haomeiwen.com/subject/esyhvftx.html