protocol buffers 序列化

作者: 持续进步者 | 来源:发表于2017-08-29 23:51 被阅读590次

什么是RPC

远程过程调用（英语：Remote Procedure Call，缩写为 RPC）是一个计算机通信协议。
该协议允许运行于一台计算机的程序调用另一台计算机的子程序，而程序员无需额外地为这个交互作用编程。

1,为了允许不同的客户端均能访问服务器，许多标准化的RPC系统应运而生了。其中大部分采用接口描述语言（Interface Description Language，IDL），方便跨平台的远程过程调用。
2,底层通过socket 传递数据
3,序列化与反序列化也叫做：编码与解码。
4,RPC 远程过程调用，很多RPC框架是跨语言的。
5，RPC 内网之间相互调用（服务于服务之间）

RPC 的开发流程

1，定义一个接口说明文件：描述了对象（结构体），对象成员，接口方法等一系列信息。
2，通过RPC框架所提供的编译器，将接口说明文件编译成具体语言文件。
3，在客户端与服务器端分别引入RPC编译器所生成的文件，即可像调用本地方法一样调用远程方法。

RPC 框架的效率

编解码的压缩比例 ，网络传输的速度

Protocol buffers

Protocol buffers是一个灵活的、高效的、自动化的机制用于序列化结构化数据。可以编译成各种源代码，比如 Java， PHP，python,nodejs,ruby ...

1,proto 文件又称为idl.
2,字段名称不用驼峰命名用下划线

示例

syntax = "proto2";

package com.lihao.netty.nettyprotobuf;

option optimize_for = SPEED;
option java_package = "com.lihao.netty.nettyprotobuf";
option java_outer_classname = "MyDataInfo";


message Person {
    required string name =1;
    required int32 age = 2;
    optional string address = 3;

}

1，syntax 表示语法如 proto2 proto3
2，package 表示包名
3，optimize_for SPEED, CODE_SIZE, or LITE_RUNTIME 表示c++ java 生成代码的生成器。
4，java_package 指定java 的包名，如果设置了java_package package将不起作用，但是还是设置。
5， java_outer_classname 表示生成的类名
6, message 表示消息体
7，required 字段描述表示此参数必须要有
8，optional 字段描述表示可选。
9，repeated 字段描述字段重复，是集合的含义，比如 list

protoc 安装和java 引入 java包

1, 下载编译器 protoc protoc-3.4.0-osx-x86_64.zip
https://github.com/google/protobuf/releases

2,设置环境变量

 vi ~/.bash_profile
 export PATH=$PATH:/Users/lixueqin/common/protoc-3.4.0/bin

3，gradle 引入文件

compile 'com.google.protobuf:protobuf-java:3.4.0'
compile 'com.google.protobuf:protobuf-java-util:3.4.0'

查看帮助

 protoc -h

.proto 文件存放位置在源代码目录也就是java 目录

Srudent.proto

syntax = "proto2";

package com.lihao.netty.protobuf;

option optimize_for = SPEED;
option java_package = "com.lihao.netty.protobuf";
option java_outer_classname = "DataInfo";


message Student {
    required string name =1;
    required int32 age = 2;
    optional string address = 3;

}

通过Srudent.proto 生成对应的java类

protoc --java_out=src/main/java src/protobuf/Student.proto

生成的 DataInfo.java 不要修改它，把他看成一个只读文件就好了.

protocol buffers 序列化测试

public class ProtoBufTest {
    public static void main(String ...arg) throws Exception {

        DataInfo.Student student = DataInfo.Student.newBuilder().setName("张三").setAge(28).setAddress("北京").build();

        System.out.println(student);

        //转换成字节可以在网络上传输
        byte[] stdent2ByteArray = student.toByteArray();

        //转换成java对象
        DataInfo.Student student2 = DataInfo.Student.parseFrom(stdent2ByteArray);

        System.out.println(student2);

    }

}

netty 对 protocol buffers 的支持

示例一：

Patients.proto

syntax = "proto3";

package com.lihao.netty.api;

option optimize_for = SPEED;
option java_package = "com.lihao.netty.api";
option java_outer_classname = "Patients";


message Patient {
    string id = 1;
    string name = 2;
    int32 age = 3;

}

//返回列表
message PatientListResponse {
    repeated Patient patientList = 1;
}

message PatientListRequest {
    string uid = 1;
}


message PatientDetailRequest {
    string uid = 1;
    string pid = 2;
}

message PatientDetailResponse {
    Patient patient = 1;
}

message Api {
    enum ApiType {
        PatientListResponseType = 0;
        PatientListRequestType = 1;
        PatientDetailRequestType = 2;
        PatientDetailResponseType = 3;

    }

    ApiType api_type = 1;

    // 一下同一时间只能访问一个
    oneof data_type {
        PatientListResponse patientListResponse = 2;

        PatientListRequest patientListRequest = 3;

        PatientDetailRequest patientDetailRequest = 4;

        PatientDetailResponse patientDetailResponse = 5;

    }


}

生成对应的java 文件

protoc --java_out=src/main/java src/protobuf/Patients.proto

server

public class ServerBuf {

    public static void main(String... arg) throws Exception {

        NioEventLoopGroup bossGroup = new NioEventLoopGroup();
        NioEventLoopGroup workGroup = new NioEventLoopGroup();

        try {

            ServerBootstrap serverBootstrap = new ServerBootstrap();

            serverBootstrap.group(bossGroup, workGroup).handler(new LoggingHandler(LogLevel.INFO))
                    .channel(NioServerSocketChannel.class).childHandler(new BufInitialzer());


            ChannelFuture channelFuture = serverBootstrap.bind(8888).sync();
            channelFuture.channel().closeFuture().sync();

        } finally {
            bossGroup.shutdownGracefully();
            workGroup.shutdownGracefully();
        }

    }

}

Initializer

public class BufInitialzer extends ChannelInitializer<SocketChannel> {
    @Override
    protected void initChannel(SocketChannel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();


        pipeline.addLast(new ProtobufVarint32LengthFieldPrepender());
        pipeline.addLast(new ProtobufVarint32FrameDecoder());
        pipeline.addLast(new ProtobufDecoder(Users.Api.getDefaultInstance()));
        pipeline.addLast(new ProtobufEncoder());

        pipeline.addLast(new BufHandler());



    }
}

handler

public class BufHandler extends SimpleChannelInboundHandler<Users.Api> {
    @Override
    protected void channelRead0(ChannelHandlerContext ctx, Users.Api msg) throws Exception {
        System.out.println("服务器接收到：");
        System.out.println(msg);

        if (msg.hasLoginRequest()) { //登陆请求

            System.out.println("------登陆请求------");

            Users.Api.Builder apiBuild = Users.Api.newBuilder();

            Users.LoginResult loginResult = Users.LoginResult.newBuilder().setUid("10001")
                    .setUsername("xiaowang").setAge(18).setAvatar("www.leyueq00.com").setToken("ewr=234sdf").build();
            apiBuild.setLoginResult(loginResult);

            Users.Api back = apiBuild.build();

            ctx.channel().writeAndFlush(back);
        }


    }
}

客户端代码

client

public class ClientBuf {
    public static void main(String... arg) throws Exception {
        NioEventLoopGroup eventLoopGroup = new NioEventLoopGroup();

        try {

            Bootstrap bootstrap = new Bootstrap();
            bootstrap.group(eventLoopGroup).channel(NioSocketChannel.class).handler(new ClientInitializer());

            ChannelFuture channelFuture = bootstrap.connect("localhost", 8888).sync();
            channelFuture.channel().closeFuture().sync();


        } finally {
            eventLoopGroup.shutdownGracefully();
        }

    }
}

Initializer

public class ClientInitializer extends ChannelInitializer<SocketChannel> {
    @Override
    protected void initChannel(SocketChannel ch) throws Exception {
        ChannelPipeline pipeline = ch.pipeline();


        pipeline.addLast(new ProtobufVarint32LengthFieldPrepender());
        pipeline.addLast(new ProtobufVarint32FrameDecoder());
        pipeline.addLast(new ProtobufDecoder(Users.Api.getDefaultInstance()));
        pipeline.addLast(new ProtobufEncoder());

        pipeline.addLast(new ClientHandler());


    }
}

handler

public class ClientHandler extends SimpleChannelInboundHandler<Users.Api> {
    @Override
    protected void channelRead0(ChannelHandlerContext ctx, Users.Api msg) throws Exception {
        System.out.println("客户端收到：");
        System.out.println(msg);

    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        Users.Api.Builder apiBuilder = Users.Api.newBuilder();

        Users.LoginRequest loginRequest = Users.LoginRequest.newBuilder().setUsername("lihao").setPassword("123").build();
        apiBuilder.setLoginRequest(loginRequest);

        Users.Api api = apiBuilder.build();

        ctx.channel().writeAndFlush(api);
    }
}

示例二使用oneOf：
Patients.proto

syntax = "proto3";

package com.lihao.netty.api;

option optimize_for = SPEED;
option java_package = "com.lihao.netty.api";
option java_outer_classname = "Patients";


message Patient {
    string id = 1;
    string name = 2;
    int32 age = 3;

}

//返回列表
message PatientListResponse {
    repeated Patient patientList = 1;
}

message PatientListRequest {
    string uid = 1;
}


message PatientDetailRequest {
    string uid = 1;
    string pid = 2;
}

message PatientDetailResponse {
    Patient patient = 1;
}

message Api {
    enum ApiType {
        PatientListResponseType = 0;
        PatientListRequestType = 1;
        PatientDetailRequestType = 2;
        PatientDetailResponseType = 3;

    }

    ApiType api_type = 1;

    // 一下同一时间只能访问一个
    oneof data_type {
        PatientListResponse patientListResponse = 2;

        PatientListRequest patientListRequest = 3;

        PatientDetailRequest patientDetailRequest = 4;

        PatientDetailResponse patientDetailResponse = 5;

    }


}

server handler

public class BufHandlerOneof extends SimpleChannelInboundHandler<Patients.Api> {

    @Override
    protected void channelRead0(ChannelHandlerContext ctx, Patients.Api msg) throws Exception {

        System.out.println("服务器收到："+msg);

        Channel channel = ctx.channel();

        switch (msg.getApiType()){
            case PatientListRequestType:

                Patients.PatientListResponse.Builder listresponseBuild = Patients.PatientListResponse.newBuilder();

                listresponseBuild.addPatientList(Patients.Patient.newBuilder().setId("10001").setName("wang xi ya").setAge(20));
                listresponseBuild.addPatientList(Patients.Patient.newBuilder().setId("10002").setName("xiao wang").setAge(24));
                listresponseBuild.addPatientList(Patients.Patient.newBuilder().setId("10003").setName("li zhi min").setAge(29));

                Patients.Api.Builder builder = Patients.Api.newBuilder().setApiType(Patients.Api.ApiType.PatientListResponseType)
                        .setPatientListResponse(listresponseBuild.build());

                channel.writeAndFlush(builder.build());

                break;

            case PatientDetailRequestType:
                Patients.PatientDetailResponse.Builder detailBuild = Patients.PatientDetailResponse.newBuilder().setPatient(Patients.Patient.newBuilder().setId("10001").setName("wang xi ya").setAge(20));
               builder = Patients.Api.newBuilder().setApiType(Patients.Api.ApiType.PatientDetailResponseType)
                        .setPatientDetailResponse(detailBuild);

                channel.writeAndFlush(builder.build());
                break;

        }
    }
}

client handler

public class ClientHandlerOneof extends SimpleChannelInboundHandler<Patients.Api> {
    @Override
    protected void channelRead0(ChannelHandlerContext ctx, Patients.Api msg) throws Exception {
        System.out.println("客户端收到：");
        System.out.println(msg);

    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {

        Random random = new Random();
        for (int i = 0; i < 5; i++) {


            int value = random.nextInt(3) <= 1 ? 1 : 2;

            System.out.println("random : " + value);

            if (value == 1) {
                Patients.Api requestlist = Patients.Api.newBuilder().setApiType(Patients.Api.ApiType.PatientListRequestType)
                        .setPatientListRequest(Patients.PatientListRequest.newBuilder().setUid("1001")).build();

                ctx.channel().writeAndFlush(requestlist);

            } else if (value == 2) {

                Patients.Api detailReuset = Patients.Api.newBuilder().setApiType(Patients.Api.ApiType.PatientDetailRequestType)
                        .setPatientDetailRequest(Patients.PatientDetailRequest.newBuilder().setPid("2001").setUid("1001")).build();

                ctx.channel().writeAndFlush(detailReuset);

            } else {
                Patients.Api requestlist = Patients.Api.newBuilder().setApiType(Patients.Api.ApiType.PatientListRequestType)
                        .setPatientListRequest(Patients.PatientListRequest.newBuilder().setUid("10012")).build();

                ctx.channel().writeAndFlush(requestlist);
            }


            System.out.println("------------------");

        }


    }
}

参照文档

https://github.com/google/protobuf/tree/master/java
https://developers.google.com/protocol-buffers/docs/proto

protocol buffers 序列化

什么是RPC

RPC 的开发流程

RPC 框架的效率

Protocol buffers

protoc 安装和java 引入 java包

查看帮助

.proto 文件存放位置在源代码目录也就是java 目录

Srudent.proto

通过Srudent.proto 生成对应的java类

protocol buffers 序列化测试

netty 对 protocol buffers 的支持

server

Initializer

handler

客户端代码

client

Initializer

handler

server handler

client handler

参照文档

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

程序员

java进阶干货

Java学习笔记

protocol buffers 序列化

什么是RPC

RPC 的开发流程

RPC 框架的效率

Protocol buffers

protoc 安装 和java 引入 java包

查看帮助

.proto 文件存放位置 在源代码目录也就是java 目录

Srudent.proto

通过Srudent.proto 生成对应的java类

protocol buffers 序列化测试

netty 对 protocol buffers 的支持

server

Initializer

handler

客户端代码

client

Initializer

handler

server handler

client handler

参照文档

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

程序员

java进阶干货

Java学习笔记

protoc 安装和java 引入 java包

.proto 文件存放位置在源代码目录也就是java 目录