Android中的网络库详解(一. HttpURLConnect

1.前言

在上一章中,讲解了计算机网络协议及通信原理,这章结合实际http网络请求框架,讲解android平台网络框架的原理及实现

2.目录

目录

3.网络库的发展史

• android应用中大都会使用Http协议来访问网络,底层网络库主要提供两种方式HttpURlConnection和HttpClient来进行Http操作
• Google为了方便开发者,在2013年Google I/O推出了新的网络通信框架Volley,可进行数据请求和图片展示
• Square公司的开源项目okhttp具有明显优势,是安卓端最火热的轻量级框架.android4.4以后,HttpURLConnection底层实现就是使用的OKhttp
• Square的右移开源项目Retrofit是一个网络封装框架,默认底层使用的是OkHttp,其使用灵活,简单,大大增加了开发效率

3.1.HttpClient

• Apache公司提供的原始http协议库
• 比较稳定,bug数量少
• api数量众多,难以升级和维护
• android6.0Google官方Api移除了HttpClient

3.2.HttpURLConnection

• 简单,容易使用和扩展
• android version 2.2之前,调用inputStream的close()方法时,会导致连接池失效,解决方法是禁用连接池.System.setProperty("http.keepAlive","false")
• android version 2.3引入了响应压缩和会话的机制,及失败重连服务器
• android version 4.0添加缓存机制

3.3.Volley

• 适合于数据量不大,通信频繁的网络操作
• 不适用于大文件下载和上传
• android version 2.3之前采用的是HttpClient,之后采用的是HttpURLConnection进行网络通信

3.4.OkHttp

• Android4.4后,HttpURLConnection底层实现使用的就是OkHttp
• 对同一主机发出的所有请求都可以共享相同的套接字连接
• 使用连接池来复用连接以提高效率
• 提供了对GZIP的默认支持来降低传输内容的大小
• 对Http响应的缓存机制,可以避免不必要的网络请求
• 当网络出现问题时,OkHttp会自动重试一个主机的多个IP地址

3.5.Retrofit

• 封装良好,使用简单,大大增加了开发效率
• 使用灵活,可以自定义底层网络请求库,自定义json解析库,可与RxJava结合使用

4.1.HttpURLConnection源码简单解析

HttpURLConnection的使用:

String urlAddress = "http://www.hao123.cn";
//分析2
URL url = new URL(urlAddress);
HttpURLConnection urlConnection = (HttpURLConnection) 
//分析1
url.openConnection();
urlConnection.setConnectTimeout(5000);
urlConnection.setReadTimeout(5000);
urlConnection.setUseCaches(true);
urlConnection.setRequestMethod("GET");
urlConnection.setRequestProperty("Content-Type","application/json");
urlConnection.addRequestProperty("Connection","Keep-Alive");
urlConnection.setRequestProperty("Charset", "gb2312");
urlConnection.connect();
if (urlConnection.getResponseCode() == 200) {
    //分析3
    String response = streamToString(urlConnection.getInputStream());
    Log.d("--TAG--", "requestGet: " + "请求成功:" + response);
    runOnUiThread(() -> mTvResponse.setText(response));
} else {
    Log.d("--TAG--", "requestGet: " + "请求失败");
}
urlConnection.disconnect();

HttpURLConnection的openConnection()实际调用的是handler的openConnection()

public URLConnection openConnection() throws java.io.IOException {
    return handler.openConnection(this);
}

再查看HttpURLConnection的创建,是通过new URL(urlAddress)

public URL(URL context, String spec, URLStreamHandler handler) throws MalformedURLException{
  ...
  //调用重载的构造器,context和handler默认为null
  if (handler == null && (handler = getURLStreamHandler(protocol)) == null) {
      throw new MalformedURLException("unknown protocol: "+protocol);
  }
  ...
  handler.parseURL(this, spec, start, limit);
  ...
}

然后查看getURLStreamHandler(protocol)

static URLStreamHandler getURLStreamHandler(String protocol) {
  ...
  if (handler == null) {
      try {
          handler = createBuiltinHandler(protocol);
      } catch (Exception e) {
          throw new AssertionError(e);
      }
  }
  ...
}

接着调用createBuiltinHandler(String protocol),创建HttpHandler

private static URLStreamHandler createBuiltinHandler(String protocol)
        throws ClassNotFoundException, InstantiationException, IllegalAccessException {
    URLStreamHandler handler = null;
    if (protocol.equals("file")) {
        handler = new sun.net.www.protocol.file.Handler();
    } else if (protocol.equals("ftp")) {
        handler = new sun.net.www.protocol.ftp.Handler();
    } else if (protocol.equals("jar")) {
        handler = new sun.net.www.protocol.jar.Handler();
    } else if (protocol.equals("http")) {
        //反射创建okhttp,实际使用的是com.squareup.okhttp包下的okHttp
        //4.3之前new HttpHandler();
        handler = (URLStreamHandler)Class.
                forName("com.android.okhttp.HttpHandler").newInstance();
    } else if (protocol.equals("https")) {
        //https加入TLS(安全传输协议),建立TCP连接后会握手
        handler = (URLStreamHandler)Class.
                forName("com.android.okhttp.HttpsHandler").newInstance();
    }
    return handler;
}

然后回到分析1,查看HttpHandler的openConnection()方法

@Override 
protected URLConnection openConnection(URL url) throws IOException {
    return newOkUrlFactory(null /* proxy */).open(url);
}

继续查看HttpHandler的newOkUrlFactory(null /* proxy */)方法

protected OkUrlFactory newOkUrlFactory(Proxy proxy) {
    OkUrlFactory okUrlFactory = createHttpOkUrlFactory(proxy);
    okUrlFactory.client().setConnectionPool(configAwareConnectionPool.get());
    return okUrlFactory;
}

接着查看createHttpOkUrlFactory(proxy)

public static OkUrlFactory createHttpOkUrlFactory(Proxy proxy) {
    OkHttpClient client = new OkHttpClient();
    //...OkHttpClient的一些配置信息
    ...
    OkUrlFactory okUrlFactory = new OkUrlFactory(client);
    ...
    return okUrlFactory;
}

然后回去查看OkUrlFactory的open(url)方法

public HttpURLConnection open(URL url) {
    return open(url,client.getProxy);
}

HttpURLConnection open(URL url, Proxy proxy) {
    String protocol = url.getProtocol();
    OkHttpClient copy = client.copyWithDefaults();
    copy.setProxy(proxy);
    //返回HttpURLConnection的具体实现类
    if (protocol.equals("http")) return new HttpURLConnectionImpl(url, copy, urlFilter);
    if (protocol.equals("https")) return new HttpsURLConnectionImpl(url, copy, urlFilter);
    throw new IllegalArgumentException("Unexpected protocol: " + protocol);
}

进行下一步查看分析3, HttpURLConnectionImpl的connect()连接方法

@Override 
public final void connect() throws IOException {
    //初始化http引擎
    initHttpEngine();
    boolean success;
    do {
           success = execute(false);
    } while (!success);
}

然后再查看execute(false)

private boolean execute(boolean readResponse) throws IOException {
  ...
  try {
    // 发起请求
    httpEngine.sendRequest();
    Connection connection = httpEngine.getConnection();
    if (connection != null) {
      route = connection.getRoute();
      handshake = connection.getHandshake();
    } else {
      route = null;
      handshake = null;
    }
    if (readResponse) {
      // 读取响应
      httpEngine.readResponse();
    }
    releaseConnection = false;
    return true;
  } 
  ...
}

查看发送请求,httpEngine.sendRequest()

public void sendRequest() throws RequestException, RouteException, IOException {
    ...
    // 当networkRequest不等于null时,需从服务端重新获取
  if (networkRequest != null) {
    // 建立Socket连接
    httpStream = connect();
    httpStream.setHttpEngine(this);
    ...
}

接着看HttpEngine的connect()方法

private HttpStream connect() throws RouteException, RequestException, IOException {
  boolean doExtensiveHealthChecks = !networkRequest.method().equals("GET");
  return streamAllocation.newStream(client.getConnectTimeout(),
      client.getReadTimeout(), client.getWriteTimeout(),
      client.getRetryOnConnectionFailure(), doExtensiveHealthChecks);
}

然后查看StreamAllocation的newStream()方法

public HttpStream newStream(int connectTimeout, int readTimeout, int writeTimeout,
    boolean connectionRetryEnabled, boolean doExtensiveHealthChecks)
    throws RouteException, IOException {
  try {
    // 寻找一个健康的连接
    RealConnection resultConnection = findHealthyConnection(connectTimeout, readTimeout,
        writeTimeout, connectionRetryEnabled, doExtensiveHealthChecks);

    // 在寻找到的连接上设置HttpStream，HttpStream是用来发送请求和接收响应的
    HttpStream resultStream;
    if (resultConnection.framedConnection != null) { // 对应于HTTP/2协议
        resultStream = new Http2xStream(this, resultConnection.framedConnection);
    } else { // 对应于HTTP/1.x协议
      resultConnection.getSocket().setSoTimeout(readTimeout);
      resultConnection.source.timeout().timeout(readTimeout, MILLISECONDS);
      resultConnection.sink.timeout().timeout(writeTimeout, MILLISECONDS);
      resultStream = new Http1xStream(this, resultConnection.source, resultConnection.sink);
    }
    synchronized (connectionPool) {
      resultConnection.streamCount++;
      stream = resultStream;
      return resultStream;
    }
  } catch (IOException e) {
    throw new RouteException(e);
  }
}

接下来看StreamAllocation的findHealthyConnection()方法

private RealConnection findHealthyConnection(int connectTimeout, int readTimeout,
    int writeTimeout, boolean connectionRetryEnabled, boolean doExtensiveHealthChecks)
    throws IOException, RouteException {
  while (true) {
    RealConnection candidate = findConnection(connectTimeout, readTimeout, writeTimeout,
        connectionRetryEnabled);
    // 如果candidate是一个全新的连接(即连接上面Steam的个数为0)，则跳过下面的健康检查直接返回。
    synchronized (connectionPool) {
      if (candidate.streamCount == 0) {
        return candidate;
      }
    }
    ...
  }
}

接着查看StreamAllocation的findConnection方法

private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout,
    boolean connectionRetryEnabled) throws IOException, RouteException {
  ...
  // RouteSelector的next方法中首先获取proxies中的第一个代理(在没有设置代理的情况下，
  // 该代理为Proxy.NO_PROXY)，然后用该代理创建Route实例
  Route route = routeSelector.next();
  RealConnection newConnection = new RealConnection(route);
  acquire(newConnection);
  // 将新创建的连接放到连接池中以备后用
  synchronized (connectionPool) {
    Internal.instance.put(connectionPool, newConnection);
    this.connection = newConnection;
    if (canceled) throw new IOException("Canceled");
  }
  // 通过新创建的连接与服务端建立Socket连接
  newConnection.connect(connectTimeout, readTimeout, writeTimeout, address.getConnectionSpecs(),
      connectionRetryEnabled);
  routeDatabase().connected(newConnection.getRoute());
  return newConnection;
}

接下来继续查看RealConnection的connect()方法

public void connect(int connectTimeout, int readTimeout, int writeTimeout,
    List<ConnectionSpec> connectionSpecs, boolean connectionRetryEnabled) throws RouteException {
    ...
  while (protocol == null) {
    try {
      // 根据上面findConnection的注释可知，默认情况下不设置代理，即proxy.type() == Proxy.Type.DIRECT是成立的，
      // 因此通过address.getSocketFactory().createSocket()创建一个Socket实例
      rawSocket = proxy.type() == Proxy.Type.DIRECT || proxy.type() == Proxy.Type.HTTP
          ? address.getSocketFactory().createSocket()
          : new Socket(proxy);
      // 利用rawSocket实例根据指定host和port发起与服务端的TCP连接
      connectSocket(connectTimeout, readTimeout, writeTimeout, connectionSpecSelector);
    } catch (IOException e) {
      ...
    }
  }
}

接下来就来看看RealConnection的connectSocket()方法

private void connectSocket(int connectTimeout, int readTimeout, int writeTimeout,
    ConnectionSpecSelector connectionSpecSelector) throws IOException {
  rawSocket.setSoTimeout(readTimeout);
  try {
    // 利用rawSocket实例根据指定host和port的服务端建立TCP连接
    Platform.get().connectSocket(rawSocket, route.getSocketAddress(), connectTimeout);
  } catch (ConnectException e) {
    throw new ConnectException("Failed to connect to " + route.getSocketAddress());
  }
  // 这两个实例就是用来读取响应和发送请求的
  source = Okio.buffer(Okio.source(rawSocket));
  sink = Okio.buffer(Okio.sink(rawSocket));
  // 只有scheme为https的情况下，下面条件才会成立
  if (route.getAddress().getSslSocketFactory() != null) {
    // 进行TLS握手
    connectTls(readTimeout, writeTimeout, connectionSpecSelector);
  } else { // 对于scheme为http情况下，则将协议版本设置为HTTP/1.1
    protocol = Protocol.HTTP_1_1;
    socket = rawSocket;
  }
  ...
  }
}

最后查看Platform的connectSocket()方法,建立TCP连接

public void connectSocket(Scoket socket, InetSocketAdress adress, int connectTimeout) throws IOException {
    socket.connect(address, connectTimeout);
}

然后再看分析3,HttpURLConnectionImpl的getInputStream()方法

@Override 
public final InputStream getInputStream() throws IOException {
  ...
  HttpEngine response = getResponse();
  ...
  return response.getResponse().body().byteStream();
}

接着查看HttpURLConnectionImpl中的getResponse()方法

private HttpEngine getResponse() throws IOException {
  initHttpEngine();
  if (httpEngine.hasResponse()) {
    return httpEngine;
  }
  while (true) {
    if (!execute(true)) {
      continue;
    }
    ...
  }
}

然后再次查看HttpURLConnectionImpl中的execute()方法

private boolean execute(boolean readResponse) throws IOException {
    ...
    if (readResponse) {
      httpEngine.readResponse();
    }
    ...
}

http的缓存策略

缓存过程发生在发送请求和处理响应中,对应HttpEngine的sendRequest()和readResponse()方法

public void sendRequest() throws RequestException, RouteException, IOException {
  if (cacheStrategy != null) return; // Already sent.
  if (httpStream != null) throw new IllegalStateException();
  // 为请求添加默认的请求头
  Request request = networkRequest(userRequest);
   //获取client的Cache,同时Cache在初始化的时候会去读取缓存目录中关于曾经请求过得所有信息
  InternalCache responseCache = Internal.instance.internalCache(client);
  Response cacheCandidate = responseCache != null
      ? responseCache.get(request)
      : null;
  long now = System.currentTimeMillis();
  // 给出请求和请求对应的缓存的响应，然后根据缓存策略得出符合缓存策略的请求和响应
  cacheStrategy = new CacheStrategy.Factory(now, request, cacheCandidate).get();
  // 当得出的请求不为null时(即networkRequest不为null)，代表该请求对应的缓存的响应不存在或者已经过期，
  // 此时需要从服务端获取，否则使用请求对应的缓存的响应
  networkRequest = cacheStrategy.networkRequest;
  cacheResponse = cacheStrategy.cacheResponse;

  if (responseCache != null) {
    responseCache.trackResponse(cacheStrategy);
  }

  if (cacheCandidate != null && cacheResponse == null) {
    closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
  }
  // 当networkRequest不等于null时，代表该请求对应的缓存响应不存在或者已经过期，需要从新从服务端获取
  if (networkRequest != null) {
    // 建立Socket连接
    httpStream = connect();
    httpStream.setHttpEngine(this);
  ...
  }
}

public void readResponse() throws IOException {
  ...
  Response networkResponse;

  if (forWebSocket) {
    httpStream.writeRequestHeaders(networkRequest);
    networkResponse = readNetworkResponse();
  } else if (!callerWritesRequestBody) {
    networkResponse = new NetworkInterceptorChain(0, networkRequest).proceed(networkRequest);
  } else {
    // 由第13步中的中newHttpEngine方法可知，forWebSocket为false、callerWritesRequestBody为true，因此会运行到这里
    // 将bufferedRequestBody中剩下的数据(即不够一个Segment的数据)全部移动到requestBodyOut中。
    if (bufferedRequestBody != null && bufferedRequestBody.buffer().size() > 0) {
      bufferedRequestBody.emit();
    }

    if (sentRequestMillis == -1) { 
      // sentRequestMillis等于-1代表请求头还没有写入到第21步中的提到的sink字段，没有写入的原因是请求体的长度是未知的。
      if (OkHeaders.contentLength(networkRequest) == -1
          && requestBodyOut instanceof RetryableSink) {
        long contentLength = ((RetryableSink) requestBodyOut).contentLength();
        networkRequest = networkRequest.newBuilder()
            // 添加头字段Content-Length，用于告诉服务端请求体的长度
            .header("Content-Length", Long.toString(contentLength))
            .build();
      }
      // writeRequestHeaders最终将请求头写入到RealBufferedSink实例中，该RealBufferedSink实例是下面第21步中的提到的sink字段，对应第33步
      httpStream.writeRequestHeaders(networkRequest);
    }

    if (requestBodyOut != null) {
      if (bufferedRequestBody != null) {
        // This also closes the wrapped requestBodyOut.
        bufferedRequestBody.close();
      } else {
        requestBodyOut.close();
      }
      if (requestBodyOut instanceof RetryableSink) {
        // 由第15步可知，当requestBodyOut的类型是RetryableSink时，写入到requestBodyOut中的请求体数据是保持在内存中，下面的方法就是将内存中的请求体数据写入到第21步中的提到的sink字段中，对应第34步。
        httpStream.writeRequestBody((RetryableSink) requestBodyOut);
      }
    }

    // 从服务端读取响应
    networkResponse = readNetworkResponse();
  }

  receiveHeaders(networkResponse.headers());

  // If we have a cache response too, then we're doing a conditional get.
  if (cacheResponse != null) {
    if (validate(cacheResponse, networkResponse)) {
      // 执行到这里，说明过期的缓存的响应仍然是可用的，这时直接使用缓存的响应,关闭网络连接,释放连接
      userResponse = cacheResponse.newBuilder()
          .request(userRequest)
          .priorResponse(stripBody(priorResponse))
          // combine方法用于更新缓存的响应的新鲜度，比如修改Last-Modified字段，对应2.3中的第7步
          .headers(combine(cacheResponse.headers(), networkResponse.headers()))
          .cacheResponse(stripBody(cacheResponse))
          .networkResponse(stripBody(networkResponse))
          .build();
      networkResponse.body().close();
      releaseStreamAllocation();

      // Update the cache after combining headers but before stripping the
      // Content-Encoding header (as performed by initContentStream()).
      InternalCache responseCache = Internal.instance.internalCache(client);
      responseCache.trackConditionalCacheHit();
      //更新缓存
      responseCache.update(cacheResponse, stripBody(userResponse));
      //解压缩response
      userResponse = unzip(userResponse);
      return;
    } else {
      closeQuietly(cacheResponse.body());
    }
  }
  // 运行到这里，说明缓存响应已经不新鲜了，这时使用networkResponse，对应于2.3中第5步否定的情况
  userResponse = networkResponse.newBuilder()
      .request(userRequest)
      .priorResponse(stripBody(priorResponse))
      .cacheResponse(stripBody(cacheResponse))
      .networkResponse(stripBody(networkResponse))
      .build();

  if (hasBody(userResponse)) {
    // maybeCache是用于对网络响应的缓存处理,只有有响应体的响应才会被缓存
    maybeCache();
    userResponse = unzip(cacheWritingResponse(storeRequest, userResponse));
  }
}

上面就是HttpURLConnection的大致流程:

• 通过Socket接口与服务端建立TCP连接
• 确定应用层协议,https需TLS握手
• TCP连接发生请求和接受响应

5.总结

这一章主要讲解了android平台的网络编程库,包括底层网络框架,HttpClient,HttpURLConnection,OkHttp,网络封装库Volley和Retrofit,也对HttpURLConnection的底层实现进行了简单分析,由于篇幅有限,下一节将继续分析后面开源库的底层实现

---

参考:
https://www.jianshu.com/p/35ecbc09c160