背景

最近在项目测试，经常出现 qa 发现客户端有 bug 后，由于没有链接 charles 代理，无法定位是客户端问题还是服务端问题。所以最近做了一款轻量的网络请求抓取 应用，用于抓取网络请求所有数据以及后续可实现请求数据的修改以及过滤。

实现方案

使用开源库 littleProxy

使用

• 在项目中添加依赖

 // littleproxy及其依赖
    implementation('net.lightbody.bmp:littleproxy:1.1.0-beta-bmp-17') {
        exclude group: 'io.netty'
    }
    implementation files('libs/netty-all-android-4.0.44.Final.jar')

• 核心代码
  启动代理类 CustomProxyServer.java

package e.lengku8e.myapplication.catchnet;

import org.littleshoot.proxy.HttpFilters;
import org.littleshoot.proxy.HttpFiltersSource;
import org.littleshoot.proxy.HttpProxyServer;
import org.littleshoot.proxy.HttpProxyServerBootstrap;
import org.littleshoot.proxy.impl.DefaultHttpProxyServer;

import java.util.ArrayList;

import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.http.HttpRequest;

public class CustomProxyServer {

    private static volatile CustomProxyServer mInstance;

    private CustomProxyServer() {
    }

    private HttpProxyServer httpProxyServer;

    public static CustomProxyServer getInstance() {
        if (mInstance == null) {
            synchronized (CustomProxyServer.class) {
                if (mInstance == null) {
                    mInstance = new CustomProxyServer();
                }
            }
        }
        return mInstance;
    }

    public ArrayList<MyResponse> responseArrayList = new ArrayList<>();

    public void start(int port) {
        try {
            HttpProxyServerBootstrap bootstrap = DefaultHttpProxyServer.bootstrap().withPort(port)
                    .withFiltersSource(new HttpFiltersSource() {
                        @Override
                        public HttpFilters filterRequest(HttpRequest originalRequest, ChannelHandlerContext channelHandlerContext) {
                            return new MyHttpFiltersAdapter(CustomProxyServer.this, originalRequest, channelHandlerContext, responseArrayList);
                        }

                        @Override
                        public int getMaximumRequestBufferSizeInBytes() {
                            return 20 * 1024 * 1024; // 每个网络请求和响应都是分次传输的Buf容量)
                        }

                        @Override
                        public int getMaximumResponseBufferSizeInBytes() {
                            return 20 * 1024 * 1024;
                        }
                    });
            httpProxyServer = bootstrap.start();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public void stop() {
        if (httpProxyServer != null) {
            httpProxyServer.stop();
        }

    }
}

请求 filter adapter 类
MyHttpFiltersAdapter.java

public class MyHttpFiltersAdapter extends HttpFiltersAdapter {
 @Override
    public HttpObject proxyToClientResponse(HttpObject httpObject) {
        Log.d("MyHttpFiltersAdapter", "proxyToClientResponse");
        return super.proxyToClientResponse(httpObject);
    }

    @Override
    public HttpResponse proxyToServerRequest(HttpObject httpObject) {
        Log.d("MyHttpFiltersAdapter", "proxyToServerRequest");
        return super.proxyToServerRequest(httpObject);
    }
    @Override
    public HttpResponse clientToProxyRequest(HttpObject httpObject) {
        Log.d("MyHttpFiltersAdapter", "clientToProxyRequest");
        return null;
    }
    @Override
    public HttpObject serverToProxyResponse(HttpObject httpObject) {
        Log.d("MyHttpFiltersAdapter", "serverToProxyResponse");
        return httpObject;
}

这里只介绍下HttpFiltersAdapter的4个关键性函数的作用

• clientToProxyRequest
  这个函数中可以获取请求的uri以及header以及 post 中的参数， 默认返回null，表示不拦截，若返回数据，则不再走proxyToClientResponse和serverToProxyResponse,这个函数中可以修改返回的数据
• proxyToServerRequest
  这个函数和上边的函数类似
• serverToProxyResponse
  这里默认返回传入参数，这里可以对返回值做一定的修改
• proxyToClientResponse
  这里默认返回传入参数，可以做一定的修改

发送一条网络请求并接收返回结构，打印调用的日志如下

06-18 15:50:37.118 28402 28560 D MyHttpFiltersAdapter: clientToProxyRequest
06-18 15:50:37.120 28402 28560 D MyHttpFiltersAdapter: proxyToServerRequest
06-18 15:50:37.159 28402 28517 D MyHttpFiltersAdapter: serverToProxyResponse
06-18 15:50:37.160 28402 28517 D MyHttpFiltersAdapter: proxyToClientResponse

遗留问题

1. 上边的实现目前只支持了 http 请求的抓取，如果想要抓 https则还需要对 https 解包，然后再生成自己的ssl正数进行加密签名。https 的抓取后续会继续进行研究。