新闻类App （MVP + RxJava + Retrofit+

Github地址：新闻类App （MVP + RxJava + Retrofit+Dagger+ARouter）

概述

• 网络的优化的维度：多维
• 仅仅重视流量不够（流量只是其中一个维度）
• 网络流量的消耗量：需要精确
• 网络相关监控：应该全面
• 粗粒度监控不能帮助我们发现，解决深层次的问题

网络优化维度

流量消耗

• 一段时间流量消耗的精准度量，网络类型，前后台
• 监控相关：用户流量消耗均值、异常率(消耗多，次数多)
• 完整链路全部监控(Request,Response),主动上报

网络请求质量

• 用户体验：请求速度，成功率
• 监控相关：请求时长，业务成功率，失败率，Top失败接口

误区

• 只关注流量消耗，忽视其它维度
• 只关注均值，整体，忽视个体

网络工具

Network Profiler

• 显示实时网络活动：发送，接受数据及连接数
• 需要开启高级分析
• 只支持httpURLConnection和okhttp网络库

• 使用：需要开启高级分析

  
  image.png
  

  勾选上enable advanced这项选择

  
  image.png

不用的时候记得关闭这选项，因为可能会影响你的构建，还有勾选完之后需要重新运行app,点击NetworkProfiler之后请求网络会出现一个三角形的形状，然后拖动选择

image.png

点击最下方这项

image.png

抓包工具

• charles
• Fiddler
• Wireshark
• TcpDump

Stetho

• 强大的应用调式桥，连接Android和Chrome
• 网络监控，试图参看，数据库参看，命令行扩展
• 使用
  github:https://github.com/facebook/stetho
  依赖

implementation 'com.facebook.stetho:stetho-okhttp3:1.5.1'

Application中oncreate方法

 Stetho.initializeWithDefaults(this);

OkHttp构建过程中

addNetworkInterceptor(new StethoInterceptor())

Chrome浏览器：chrome://inspect

精准获取流量消耗

如何判断App流量消耗偏高

• 绝对值看不出高低
• 对比竞品，相同case对比流量消耗
• 异常监控超过正常指标

测试方案

• 设置——流量管理
• 抓包工具：只允许本APP联网
• 可以解决绝大数问题，但是线上场景，线下可能遇不到

线上流量获取方案

TrafficStats：不采用:

• API8以上手机上次重启以来的流量数据统计
• 缺点：无法获取某个时间段内的流量消耗

tatic long  getMobileRxBytes()  //获取通过移动数据网络收到的字节总数
static long  getMobileTxBytes()  //通过移动数据网发送的总字节数  
static long  getTotalRxBytes()  //获取设备总的接收字节数 
static long  getTotalTxBytes()  //获取设备总的发送字节数
static long  getUidRxBytes(int uid)  //获取指定uid的接收字节数  
static long  getUidTxBytes(int uid) //获取指定uid的发送字节数 

NetworkStatsManager

• API23之后的流量统计
• 可获取指定时间间隔内的流量信息
• 可获取不同网络类型下的消耗
• 代码,注意需要打开权限，否则程序会报错并需要添加权限

    <uses-permission android:name="android.permission.READ_PHONE_STATE"/>
    <uses-permission android:name="android.permission.PACKAGE_USAGE_STATS" tools:ignore="ProtectedPermissions"/>

  /**
     *  打开“有权查看使用情况的应用”页面
     */
    private boolean hasPermissionToReadNetworkStats() {
        if (Build.VERSION.SDK_INT < Build.VERSION_CODES.M) {
            return true;
        }
        final AppOpsManager appOps = (AppOpsManager) getSystemService(Context.APP_OPS_SERVICE);
        int mode = appOps.checkOpNoThrow(AppOpsManager.OPSTR_GET_USAGE_STATS,
                android.os.Process.myUid(), getPackageName());
        if (mode == AppOpsManager.MODE_ALLOWED) {
            return true;
        }

        requestReadNetworkStats();
        return false;
    }

   
    private void requestReadNetworkStats() {
        Intent intent = new Intent(Settings.ACTION_USAGE_ACCESS_SETTINGS);
        startActivity(intent);
    }

    private void getNetStatus(long startTime, long endTime) {
        if (Build.VERSION.SDK_INT < Build.VERSION_CODES.M) {
            return;
        }
        try {
            long total = 0;
            //发送和接受流量
            long netDataRx = 0;//接受
            long netDataTx = 0;//发送
            // 获取subscriberId
            TelephonyManager telecomManager = (TelephonyManager) getSystemService(TELEPHONY_SERVICE);
            String subscriberId = telecomManager.getSubscriberId();
            NetworkStatsManager manager = (NetworkStatsManager) getSystemService(NETWORK_STATS_SERVICE);
            //设置本月的第一天为开始时间
            NetworkStats networkStats = null;
            NetworkStats.Bucket bucket = new NetworkStats.Bucket();
            networkStats = manager.querySummary(NetworkCapabilities.TRANSPORT_WIFI, subscriberId, startTime, endTime);
            do {
                networkStats.getNextBucket(bucket);
                int summaryUid = bucket.getUid();
                if (getUidByPackageName() == summaryUid) {
                    netDataRx += bucket.getRxBytes();
                    netDataTx += bucket.getTxBytes();
                }
                Log.i(MainActivity.class.getSimpleName(), "uid:" + bucket.getUid() + " rx:" + bucket.getRxBytes() +
                        " tx:" + bucket.getTxBytes());
                total += bucket.getRxBytes() + bucket.getTxBytes();
            } while (networkStats.hasNextBucket());
            LogUtils.e("gankzhihu app net cost" + total);
        } catch (RemoteException e) {
            e.printStackTrace();
        }
    }

    public int getUidByPackageName() {
        int uid = -1;
        PackageManager packageManager = getPackageManager();

        PackageInfo packageInfo = null;
        try {
           //包名
            packageInfo = packageManager.getPackageInfo("com.peakmain.gankzhihu", PackageManager.GET_META_DATA);
        } catch (PackageManager.NameNotFoundException e) {
            e.printStackTrace();
        }

        assert packageInfo != null;
        uid = packageInfo.applicationInfo.uid;
        Log.e(MainActivity.class.getSimpleName(), packageInfo.packageName + " uid:" + uid);
        return uid;
    }

前后台流量获取方案

• 线上反馈App后台跑流量
• 只获取一个时间段值不够
• 方案
  后台定时任务->获取间隔内流量->记录前后台->分别计算->上报APM后台->流量治理依据

      if (hasPermissionToReadNetworkStats()) {
            Executors.newScheduledThreadPool(1).schedule(new Runnable() {
                @Override
                public void run() {
                   long netUse= getNetStatus(getTimesMonthMorning()-30*1000, System.currentTimeMillis());
                   //当前是前后还是后台，30s消耗的流量                    
                }
            }  ,30, TimeUnit.SECONDS);
            getNetStatus(getTimesMonthMorning(), System.currentTimeMillis());
        }

网络请求流量优化

使用网络场景

• 数据：API，资源包(升级包，H5、配置信息
• 图片：下载，上传
• 监控:APM相关、单点问题相关

数据缓存

• 服务器返回加上过期时间，避免每次重新获取
• 节约流量且大幅度提高数据的访问速度，更好的用户体验
• OKHttp、Volley都有较好的实战
• 代码：RetrofitManager

public class RetrofitManager {
    //连接超时
    private static long CONNECT_TIMEOUT = 60L;
    //阅读超时
    private static long READ_TIMEOUT = 10L;
    //写入超时
    private static long WRITE_TIMEOUT = 10L;
    //设缓存有效期为1天
    private static final long CACHE_STALE_SEC = 60 * 60 * 24 * 1;
    //查询缓存的Cache-Control设置，为only-if-cached时只查询缓存而不会请求服务器，max-stale可以配合设置缓存失效时间
    public static final String CACHE_CONTROL_CACHE = "only-if-cached, max-stale=" + CACHE_STALE_SEC;
    //查询网络的Cache-Control设置
    //(假如请求了服务器并在a时刻返回响应结果，则在max-age规定的秒数内，浏览器将不会发送对应的请求到服务器，数据由缓存直接返回)
    public static final String CACHE_CONTROL_NETWORK = "Cache-Control: public, max-age=10";
    // 避免出现 HTTP 403 Forbidden，参考：http://stackoverflow.com/questions/13670692/403-forbidden-with-java-but-not-web-browser
    private static final String AVOID_HTTP403_FORBIDDEN = "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11";
    private static volatile OkHttpClient mOkHttpClient;
    /**
     * 云端响应头拦截器，用来配置缓存策略
     */
    private static final Interceptor mRewriteCacheControlInterceptor = new Interceptor() {
        @Override
        public Response intercept(Chain chain) throws IOException {
            Request request = chain.request();//获得上一个请求
            if (!NetworkUtils.isConnected()) {
                request = request.newBuilder()
                        .cacheControl(CacheControl.FORCE_CACHE)
                        .build();
            }
            Response originalResponse = chain.proceed(request);
            if (NetworkUtils.isConnected()) {
                //有网的时候读接口上的@Headers里的配置，可以在这里进行统一的设置
                String cacheControl = request.cacheControl().toString();
                return originalResponse.newBuilder()
                        .header("Cache-Control", cacheControl)
                        .removeHeader("Pragma")
                        .build();
            } else {
                return originalResponse.newBuilder()
                        .header("Cache-Control", "public, only-if-cached, max-stale=" + CACHE_CONTROL_CACHE)
                        .removeHeader("Pragma")
                        .build();
            }
        }
    };
    private static final HttpLoggingInterceptor mLoggingInterceptor = new HttpLoggingInterceptor()
            .setLevel(HttpLoggingInterceptor.Level.BODY);
    /**
     * 日志拦截器
     */
    private static final Interceptor mLoggingIntercepter = new Interceptor() {
        @Override
        public Response intercept(Chain chain) throws IOException {
            Request request = chain.request();
            Response response = chain.proceed(request);
            String isSuccess = response.isSuccessful() ? "true" : "false";
            Logger.w(isSuccess);
            ResponseBody body = response.body();
            BufferedSource source = body.source();
            source.request(Long.MAX_VALUE);
            Buffer buffer = source.buffer();
            Charset charset = Charset.defaultCharset();
            MediaType contentType = body.contentType();
            if (contentType != null) {
                charset = contentType.charset();
            }
            String bodyString = buffer.clone().readString(charset);
            Logger.w(String.format("Received response json string " + bodyString));
            return response;
        }
    };

    /**
     * 获取OkHttpClient实例
     *
     * @return
     */
    private static OkHttpClient getOkHttpClient() {
        if (mOkHttpClient == null) {
            synchronized (RetrofitManager.class) {
                Cache cache = new Cache(new File(App.getAppContext().getCacheDir(), "HttpCache"), 1024 * 1024 * 100);
                if (mOkHttpClient == null) {
                    mOkHttpClient = new OkHttpClient.Builder()
                            .cache(cache)
                            .connectTimeout(CONNECT_TIMEOUT, TimeUnit.SECONDS)
                            .readTimeout(READ_TIMEOUT, TimeUnit.SECONDS)
                            .writeTimeout(WRITE_TIMEOUT, TimeUnit.SECONDS)
                            .addInterceptor(mRewriteCacheControlInterceptor)
                            .addNetworkInterceptor(new StethoInterceptor())
                            //.addInterceptor(mLoggingIntercepter)
//                            .addInterceptor(interceptor)
//                            .cookieJar(new CookiesManager())
//                            .cookieJar(cookieJar)
                            .build();
                }
            }
        }
        return mOkHttpClient;
    }

    /**
     * 获取玩android的service
     */
    public static <T> T create(Class<T> clazz) {
        Retrofit retrofit = new Retrofit.Builder().baseUrl(Constant.REQUEST_BASE_URL)
                .client(getOkHttpClient())
                .addCallAdapterFactory(RxJava2CallAdapterFactory.create())
                .addConverterFactory(GsonConverterFactory.create())
                .build();
        return retrofit.create(clazz);
    }

}

okhttp对post请求的数据不缓存

增量数据更新

• 加上版本的概念，只传输有变化的数据
• 配置信息、省市区县更新

数据压缩

• Post请求Body使用GZip
• 请求头压缩
• 图片上传之前必须压缩
• 图片压缩库：Luban
  github:https://github.com/Curzibn/Luban
  依赖

implementation 'top.zibin:Luban:1.1.8'

图片优化

• 图片使用策略：优先使用缩略图
  推荐工具网站：https://yijiangaitu.com
• 使用webP格式的图片

网络请求质量优化

质量指标

• 网络请求速度
• 网络请求成功率

http请求过程

• 请求到达运营商的Dns服务器并解析成对应的IP地址
• 创建连接，根据IP地址找到相应的服务器，发起一个请求
• 服务器找到对应的资源原路返回访问的用户

DNS相关

• 问题：DNS被劫持，DNS解析慢
• 方案：使用HttpDNS,绕过运营商域名解析的过程
• 优势：降低平均访问时长，提高连接成功率
• https单域名解析
  文档：https://help.aliyun.com/document_detail/30140.html?spm=a2c4g.11186623.2.42.202b13e8Wj7LEl
  依赖

    maven {
            url 'http://maven.aliyun.com/nexus/content/repositories/releases/'
        }
    compile ('com.aliyun.ams:alicloud-android-httpdns:1.1.7@aar') {
        transitive true
    }

• 代码

public class OkHttpDNS implements Dns {
    private HttpDnsService mDnsService;
    private static OkHttpDNS instance = null;
    private OkHttpDNS(Context context) {
        //第二个参数阿里云的id,不可为空
        mDnsService = HttpDns.getService(context, "");
    }

    public static OkHttpDNS getInstance(Context context) {
        if (instance == null) {
            synchronized (OkHttpDNS.class) {
                if (instance == null) {
                    instance = new OkHttpDNS(context);
                }
            }
        }
        return instance;
    }
    @Override
    public List<InetAddress> lookup(String hostname) throws UnknownHostException {
        String ip = mDnsService.getIpByHostAsync(hostname);
        if(ip!=null){
            List<InetAddress> inetAddresses = Arrays.asList(InetAddress.getAllByName(ip));
            return inetAddresses;
        }
        //系统的DNS
        return Dns.SYSTEM.lookup(hostname);
    }
}

使用在okhttp构建过程中添加.dns选项

http协议版本历史

• 1.0:版本TCP连接不复用
• 1.1：引入持久连接，但数据通讯按次序进行
• 2:多工。客户端，服务器双向实时通信

网络请求质量监控

• 接口请求耗时，成功率，错误码
• 图片加载每一步耗时

public class OkHttpEventListener extends EventListener {
    public static final Factory FACTORY=new Factory() {
        @Override
        public EventListener create(Call call) {
            return new OkHttpEventListener();
        }
    };
   private OkHttpEvent mOkHttpEvent;
    public OkHttpEventListener() {
        super();
        mOkHttpEvent=new OkHttpEvent();
    }

    @Override
    public void callStart(Call call) {
        super.callStart(call);

    }

    @Override
    public void dnsStart(Call call, String domainName) {
        super.dnsStart(call, domainName);
        mOkHttpEvent.dnsStartTime=System.currentTimeMillis();
    }

    @Override
    public void dnsEnd(Call call, String domainName, List<InetAddress> inetAddressList) {
        super.dnsEnd(call, domainName, inetAddressList);
        mOkHttpEvent.dnsEndTime=System.currentTimeMillis();
    }

    @Override
    public void connectStart(Call call, InetSocketAddress inetSocketAddress, Proxy proxy) {
        super.connectStart(call, inetSocketAddress, proxy);
    }

    @Override
    public void secureConnectStart(Call call) {
        super.secureConnectStart(call);
    }

    @Override
    public void secureConnectEnd(Call call, @Nullable Handshake handshake) {
        super.secureConnectEnd(call, handshake);
    }

    @Override
    public void connectEnd(Call call, InetSocketAddress inetSocketAddress, Proxy proxy, @Nullable Protocol protocol) {
        super.connectEnd(call, inetSocketAddress, proxy, protocol);
    }

    @Override
    public void connectFailed(Call call, InetSocketAddress inetSocketAddress, Proxy proxy, @Nullable Protocol protocol, IOException ioe) {
        super.connectFailed(call, inetSocketAddress, proxy, protocol, ioe);
    }

    @Override
    public void connectionAcquired(Call call, Connection connection) {
        super.connectionAcquired(call, connection);
    }

    @Override
    public void connectionReleased(Call call, Connection connection) {
        super.connectionReleased(call, connection);
    }

    @Override
    public void requestHeadersStart(Call call) {
        super.requestHeadersStart(call);
    }

    @Override
    public void requestHeadersEnd(Call call, Request request) {
        super.requestHeadersEnd(call, request);
    }

    @Override
    public void requestBodyStart(Call call) {
        super.requestBodyStart(call);
    }

    @Override
    public void requestBodyEnd(Call call, long byteCount) {
        super.requestBodyEnd(call, byteCount);
    }

    @Override
    public void responseHeadersStart(Call call) {
        super.responseHeadersStart(call);
    }

    @Override
    public void responseHeadersEnd(Call call, Response response) {
        super.responseHeadersEnd(call, response);
    }

    @Override
    public void responseBodyStart(Call call) {
        super.responseBodyStart(call);
    }

    @Override
    public void responseBodyEnd(Call call, long byteCount) {
        super.responseBodyEnd(call, byteCount);
    }

    @Override
    public void callEnd(Call call) {
        super.callEnd(call);
        mOkHttpEvent.apiSuccess=true;
    }

    @Override
    public void callFailed(Call call, IOException ioe) {
        super.callFailed(call, ioe);
        mOkHttpEvent.apiSuccess=false;
        mOkHttpEvent.errReson= Log.getStackTraceString(ioe);
        LogUtils.e("reason"+mOkHttpEvent.errReson);
    }
}

OkHttpEvent实体类

public class OkHttpEvent {
    public long dnsStartTime;
    public long dnsEndTime;
    public long responseBodySize;
    //判断有没有成功
    public boolean apiSuccess;
    public String errReson;
}

使用okhttp构建的时候添加.eventListenerFactory(OkHttpEventListener.FACTORY)这项

• 获取图片每步耗时，以Fresco为例子

public class FrescoTraceListener implements RequestListener {

    @Override
    public void onRequestStart(ImageRequest request, Object callerContext, String requestId, boolean isPrefetch) {

    }

    @Override
    public void onRequestSuccess(ImageRequest request, String requestId, boolean isPrefetch) {

    }

    @Override
    public void onRequestFailure(ImageRequest request, String requestId, Throwable throwable, boolean isPrefetch) {

    }

    @Override
    public void onRequestCancellation(String requestId) {

    }

    @Override
    public void onProducerStart(String requestId, String producerName) {

    }

    @Override
    public void onProducerEvent(String requestId, String producerName, String eventName) {

    }

    @Override
    public void onProducerFinishWithSuccess(String requestId, String producerName, @Nullable Map<String, String> extraMap) {

    }

    @Override
    public void onProducerFinishWithFailure(String requestId, String producerName, Throwable t, @Nullable Map<String, String> extraMap) {
    }

    @Override
    public void onProducerFinishWithCancellation(String requestId, String producerName, @Nullable Map<String, String> extraMap) {
    }
    @Override
    public void onUltimateProducerReached(String requestId, String producerName, boolean successful) {
    }

    @Override
    public boolean requiresExtraMap(String requestId) {
        return false;
    }
}

初始化Fresco的时候

       Set<RequestListener> listenerset = new HashSet<>();
        listenerset.add(new FrescoTraceListener());
        ImagePipelineConfig config = 
        ImagePipelineConfig.newBuilder(mContext).setRequestListeners(listenerset)
                .build();
        Fresco.initialize(mContext,config);

其他优化

• 多次失败后一定时间内不进行请求，避免雪崩的效应
• CDN加速，提高带宽，动静资源分离(更新后清理缓存)
• 减少传输量，注意请求时机及频率

网络体系优化建设方案

线下测试

• 方案：只抓单独APP，关闭其他app请求
• 侧重点：请求有误，多余，网络切换，弱网，无网测试

线上监控

• 服务器监控
  请求耗时(区分地域，时间段，版本，机型)
  失败率(业务失败和请求失败)
  Tcp失败接口，异常接口
• 客户端监控
  接口的每一步信息(DNS,连接，请求等)
  请求次数，网络包的大小，失败原因
  图片监控

异常监控体系

• 服务器防刷：超限拒绝访问
• 客户端：大文件预警，异常兜底策略
• 单点问题追查