DSF 消息序列化与反序列解析
DSF框架消息序列化与反序列化 ProtoBuff协议 ， 是通过拓展Netty的ByteBuf实现的， 具体实现类名为com.huawei.csc.remoting.common.buf.impl.NettyBuffer
com.huawei.csc.usf.framework.bus.MsgCoder 实现了com.huawei.csc.remoting.common.encode.MessageEncoder和 com.huawei.csc.remoting.common.decode.MessageDecoder两个接口，com.huawei.csc.usf.framework.bus.DSFMsgCoder 继承了MsgCoder实现了对DSF协议的编解码器。
协议编码
对象编码方式
| ID | Type |
|---|---|
| -1 | null |
| 1 | string |
| 2 | protostaff bytes |
| 3 | USFCtxObject |
| 4 | byte[] |
string 对象编码方法：
length | UTF8 bytes
byte[] 对象编码方法 ：
length | bytes
USFCtxObject 对象编码方式
USFObjType | Obj
protostaff bytes 编码方式
length | bytes
Obj 编码方法：
id | obj
其他泛型的编码方式：
map 编码方式
mapsize | Key(string) | Value(obj) | key2|value2...
array of obj[]
length |obj1 | obj2
UsfObjType: 0 LOCAL, 1 REMOTE, 2 LOCAL_MANUAL;
|仅为了便于阅读，并未使用分割符
协议的编码方式是典型的变长编码法：所有对象的编码方式都是 id|obj的方式进行编码，先向buffer中写对象类型，然后紧跟着写对象序列化后的的bytes，中间的“|”仅为了便于阅读，并不是分隔符。string、byte[]，除sting、byte[] 和USFCtxObject 之外，其他自定义的对象都是通过protostaff进行序列化，string\byte[]\USFCtxObject、Map、Array 的序列化方法见表格，一般是 Lenght|bytes的方法进行序列化。
DSF协议定义：
| magic | msgType | msg encode(type val name -> ) |
|---|---|---|
| 4 | bindrequest | int MsgLen </P> byte 4 MsgMagic </P> int seqid </P> string payload</P> string version |
| 5 | bindresponse,bindVersionV1 | int MsgLen </P> byte 5 MsgMagic </p> int seqid </p>string PayLoad |
| 5 | bindresponse,otherversion | int MsgLen </p> byte 5 Msg Magic</p> int SeqId </p> Object PayLoad |
| 0 | request | int MsgLen</p> byte 0 MsgMagic </p> string ReqId </p>int isAsync :0-> no 1->yes </p> int 0 msgType: 0->request 1->reply </p> string serviceName </p> string operationName </p>Long Long Int : 0L0L0d MsgSeprator</p> Obj[] payload </p> string group</p> map attachement (with sendtime) </p> Object reqobj </p> Object null(0xffffffff) excption |
| 0 | reply | int MsgLen</p> byte 0 Msg Magic </p> string ReqId </p> int isAsync </p> int 1 msgType:0->request 1->reply </p> string serviceName </p> string operation </p> Long 0L |Long 0L|int 0 MsgSeprator</p> payload:null(-1,0xfffffffff) </p> string group </p> map attachement(with sendtime)</p> Object rspObj </p> Object exception |
msgtype : 0 request 1 respons
isSync : 0 no 1 yes
len-4 是因为 java int 为32位 占4个byte
报文中的 0xffffffff 是 表示对象为 null
从抓包分析来看，DSF协议并没有按照其父类的设计，写 magic、servicegroup、src、des 等等，估计是开发人员没有按照大师的设计来，进行了一部分简化。另外本身协议设计也是根据固定读写顺序来的，协议拓展性并不高；协议内容中因为大量使用int类型传输数据，协议压缩率可以进一步提升。
DSF 抓包分析
--request bigend ?
0000008b00000000013100000000000000000000000853657276696365410000000873617948656c6c6f000000000000000000000000000000000000000000000001000000010000000568656c6c6f0000000764656661756c74000000010000000873656e6454696d650000000300000000000000010000000d31353031333739303237393037ffffffffffffffff
--request hex
0 1 2 3 4 5 6 6 8 9 a b c d e f
-----------------------------------------------
00000000 00 00 00 8b 00 00 00 00 01 31 00 00 00 00 00 00 ........ .1......
00000010 00 00 00 00 00 08 53 65 72 76 69 63 65 41 00 00 ......Se rviceA..
00000020 00 08 73 61 79 48 65 6c 6c 6f 00 00 00 00 00 00 ..sayHel lo......
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ........ ........
00000040 00 01 00 00 00 01 00 00 00 05 68 65 6c 6c 6f 00 ........ ..hello.
00000050 00 00 07 64 65 66 61 75 6c 74 00 00 00 01 00 00 ...defau lt......
00000060 00 08 73 65 6e 64 54 69 6d 65 00 00 00 03 00 00 ..sendTi me......
00000070 00 00 00 00 00 01 00 00 00 0d 31 35 30 31 33 37 ........ ..150137
00000080 39 30 32 37 39 30 37 ff ff ff ff ff ff ff ff 9027907. .......
request 解析
0 1 2 3 4 5 6 7 8 9 a b c d e f ascii msg
-----------------------------------------------
00000000 00 00 00 8b|00 00 00 00| 01|31|00 00 00 00|00 00 ........ .1......
package len 8b|msg type 0 |len|id| is async |msg
00000010 00 00|00 00 00 08|53 65 72 76 69 63 65 41|00 00 ......Se rviceA..
send 0 |string len8| S e r v i c e A|str len 8
|Servicename |
00000020 00 08|73 61 79 48 65 6c 6c 6f|00 00 00 00 00 00 ..sayHel lo......
| s a y H e l l o| OL
|operation name |OL 0L 0d seprator
00000030 00 00|00 00 00 00 00 00 00 00|00 00 00 00|00 00 ........ ........
| OL | Od |obj len 1
00000040 00 01|00 00 00 01|00 00 00 05|68 65 6c 6c 6f|00 ........ ..hello.
| obj type 1|string len 5| h e l l o|
|ref objtype|in param |
00000050 00 00 07|64 65 66 61 75 6c 74|00 00 00 01|00 00 ...defau lt......
str len 7| d e f a u l t|map size 1|
|group name |attachment map
00000060 00 08|73 65 6e 64 54 69 6d 65|00 00 00 03|00 00 ..sendTi me......
str len 8| s e n d T i m e| |
|key :sendTime |USFCtxObject|USFOBjType
00000070 00 00|00 00 00 01|00 00 00 0d|31 35 30 31 33 37 ........ ..150137
|string 1| | 1 5 0 1 3 7
|obj type |str len 10 |string content
00000080 39 30 32 37 39 30 37|ff ff ff ff |ff ff ff ff 9027907. .......
9 0 2 7 9 0 8|req obj |exception
--reply
0000008d00000000013100000000000000010000000853657276696365410000000873617948656c6c6f0000000000000000000000000000000000000000ffffffff0000000764656661756c74000000010000000873656e6454696d650000000300000000000000010000000d31353031333739303237393131000000010000000b68656c6c6f3a68656c6c6fffffffff
--reply hex
0 1 2 3 4 5 6 6 8 9 a b c d e f
00000000 00 00 00 8d|00 00 00 00| 01|31|00 00 00 00|00 00 ........ .1......
len 8d |type 0 |len|id|isAsync | msg
00000010 00 01|00 00 00 08|53 65 72 76 69 63 65 41|00 00 ......Se rviceA..
reply 01| str len |servicename |str len
00000020 00 08|73 61 79 48 65 6c 6c 6f|00 00 00 00 00 00 ..sayHel lo......
|operation name | 0L
00000030 00 00|00 00 00 00 00 00 00 00|00 00 00 00|ff ff ........ ........
| 0L | 0d |null -1 reply has in params
00000040 ff ff|00 00 00 07|64 65 66 61 75 6c 74|00 00 00 ......de fault...
| str len 7| d e f a u l t|
|groupname |attachement map size 1
00000050 01|00 00 00 08|73 65 6e 64 54 69 6d 65|00 00 00 .....sen dTime...
|str len | key |obj type usfctxobj 03
00000060 03 |00 00 00 00|00 00 00 01|00 00 00 0d|31 35 30 ........ .....150
| local |obj type string len 0d|
00000070 31 33 37 39 30 32 37 39 31 31|00 00 00 01|00 00 13790279 11......
rsptime |resp obj |
|obj type 1 |string len 0x0b
00000080 00 0b|68 65 6c 6c 6f 3a 68 65 6c 6c 6f|ff ff ff ..hello: hello...
| |Null exception
|response content |
00000090 ff .
通过以上研究，可以在后期使用跨语言访问dsf,详细抓包文件请使用 tcp.port eq 50254 ,另外可能是gitea存在bug，无法正常查看部分表格。
Next： DSF zookeeper的Node 组织
结构:
DSFCtxObject
--type : int( LOCAL,REMOTE,LOCAL_MANUAL)
--Obj : protostaff bytes
-------- length: int
-------- ObjectCompressBytes :gziped bytes,用protostaff.GraphIOUtil.writeTo 方法产生ObjectPack 的bytes，然后gzip
---ObjectPack
--Object
DSFCtxObject 解码方法：
- read type
- read length
- read byte[length]
4.unzip
5.GraphIOUtil.mergeFrom
6.gen DSFCtxObject
DSFCtxObject编码方法：
0.write byte 4
1.write DSFCtxObject.type
2.ObjectPack(DSFCtxObject.obj)
3.protostaff.GraphIOUtil.writeTo(objpack) 生成待压缩bytes
4.gzip compress
5.write length of compress bytes
6.write compressed bytes
网友评论