美文网首页
万事有商量,论沟通的重要性 (NFS篇)

万事有商量,论沟通的重要性 (NFS篇)

作者: hynoor | 来源:发表于2019-05-08 00:39 被阅读0次

    主流的网络存储协议中,几乎都有一部叫做协商 (Negotiate), 其重要作用就是把连接双方的各自的情况告知对方,以便找出一个双方都满意的方式来传输数据,这好比男女结婚前会谈一阵子朋友,双方对彼此了解以后,才能更为和谐的相处,不然就会出现婚姻的危机。

    主流的NAS网络存储协议在我看来就是NFS和SMB(CIFS), 双方的协商过程都叫做Negotiate, 这篇文章的目的就是要‘偷听’这两种协议分别在‘谈朋友’的过程中的悄悄话。

    先说NFS, 以最新的4.1版本为例, 首先要确定协商(Negotiate)发生在哪个阶段,通过RFC知道,协商一般会在mount 的时候(还有就是在NFS服务器重启后),

    二话不说,开始mount抓包吧

    [root@centos ~] tshark -t ud -Y "ip.addr==<server_ip>" -r mount-nfs4-1.pcap >>mount-nfs4-1.txt; vim mount-nfs4-1.txt
    
    871 11:52:29.533571 client -> server TCP 74 881 → 2049 [SYN] Seq=0 Win=26880 Len=0 MSS=8960 SACK_PERM=1 TSval=1968097466 TSecr=0 WS=128
    872 11:52:29.558365 server -> client TCP 74 2049 → 881 [SYN, ACK] Seq=0 Ack=1 Win=28960 Len=0 MSS=1460 SACK_PERM=1 TSval=1686911269 TSecr=1968097466 WS=128
    873 11:52:29.558557 client -> server TCP 66 881 → 2049 [ACK] Seq=1 Ack=1 Win=26880 Len=0 TSval=1968097491 TSecr=1686911269
    874 11:52:29.558608 client -> server NFS 110 V4 NULL Call
    876 11:52:29.585548 server -> client TCP 66 2049 → 881 [ACK] Seq=1 Ack=45 Win=29056 Len=0 TSval=1686911276 TSecr=1968097491
    877 11:52:29.585626 server -> client NFS 94 V4 NULL Reply (Call In 874)
    878 11:52:29.585657 client -> server TCP 66 881 → 2049 [ACK] Seq=45 Ack=29 Win=26880 Len=0 TSval=1968097518 TSecr=1686911276
    879 11:52:29.585978 client -> server NFS 362 V4 Call EXCHANGE_ID
    882 11:52:29.610601 server -> client NFS 238 V4 Reply (Call In 879) EXCHANGE_ID
    883 11:52:29.610929 client -> server NFS 322 V4 Call CREATE_SESSION
    884 11:52:29.634930 server -> client NFS 194 V4 Reply (Call In 883) CREATE_SESSION
    885 11:52:29.635147 client -> server NFS 222 V4 Call RECLAIM_COMPLETE
    886 11:52:29.659248 server -> client NFS 158 V4 Reply (Call In 885) RECLAIM_COMPLETE
    887 11:52:29.659499 client -> server NFS 230 V4 Call SECINFO_NO_NAME
    888 11:52:29.683580 server -> client NFS 258 V4 Reply (Call In 887) SECINFO_NO_NAME
    889 11:52:29.690901 client -> server NFS 242 V4 Call PUTROOTFH | GETATTR
    898 11:52:29.715565 server -> client NFS 378 V4 Reply (Call In 889) PUTROOTFH | GETATTR
    899 11:52:29.715809 client -> server NFS 302 V4 Call GETATTR FH: 0x12166020
    900 11:52:29.739614 server -> client NFS 234 V4 Reply (Call In 899) GETATTR
    

    先看summary, #871 - #873 很明显是TCP层的三次握手(顺便说一句,NFS 从version4 开始支持吃TCP的传输方式), 接下来是一个NFS NULL的call,用具检测NFS服务是否在线,接下来便要开始协商和建立会话了, 这期间主要会经历4步

    1. Frame #879 EXCHANGE_ID
    2. Frame #883 CREATE_SESSION
    3. Frame #885 RECLAIM_COMPLETE
    4. Frame #887 SECINFO_NO_NAME
    5. Frame #889 PUTROOTFH

    EXCHANGE_ID
    这一步就是告知对方自己的名字,就行连个刚见面的商人互相交换名片一样,我们展开这一帧开仔细看看他们之间说了什么:

    Network File System, Ops(1): EXCHANGE_ID
        [Program Version: 4]
        [V4 Procedure: COMPOUND (1)]
        Tag: <EMPTY>
            length: 0
            contents: <EMPTY>
        minorversion: 1
        Operations (count: 1): EXCHANGE_ID
            Opcode: EXCHANGE_ID (42)
                eia_clientowner
                    verifier: 0x5cc7bb2c294e6346    # 提供给服务器一个verifier,用于服务器生成Client ID
                    Data: <DATA>
                        length: 48
                        contents: <DATA>
                    flags: 0x00000101, EXCHGID4_FLAG_BIND_PRINC_STATEID, EXCHGID4_FLAG_SUPP_MOVED_REFER
                        0... .... .... .... .... .... .... .... = EXCHGID4_FLAG_CONFIRMED_R: Not set
                        .0.. .... .... .... .... .... .... .... = EXCHGID4_FLAG_UPD_CONFIRMED_REC_A: Not set
                        .... .... .... .0.. .... .... .... .... = EXCHGID4_FLAG_USE_PNFS_DS: Not set
                        .... .... .... ..0. .... .... .... .... = EXCHGID4_FLAG_USE_PNFS_MDS: Not set
                        .... .... .... ...0 .... .... .... .... = EXCHGID4_FLAG_USE_NON_PNFS: Not set
                        .... .... .... .... .... ...1 .... .... = EXCHGID4_FLAG_BIND_PRINC_STATEID: Set
                        .... .... .... .... .... .... .... ..0. = EXCHGID4_FLAG_SUPP_MOVED_MIGR: Not set
                        .... .... .... .... .... .... .... ...1 = EXCHGID4_FLAG_SUPP_MOVED_REFER: Set
                eia_state_protect: SP4_NONE (0)
                eia_client_impl_id
                    Implementor DNS domain name(nii_domain): kernel.org
                        length: 10
                        contents: kernel.org
                        fill bytes: opaque data
                    Implementation product name(nii_name): Linux 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64
                        length: 75
                        contents: Linux 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64
                        fill bytes: opaque data
                    Build timestamp(nii_date)
                        seconds: 0
                        nseconds: 0
        [Main Opcode: EXCHANGE_ID (42)]
    

    可以看到这一步客户端会告诉服务器一串验证码(verifier),自己的Linux 内核版本,自己的DNS信息等等等,而服务器收到后会告知其为此客户端分配的Client ID 这个信息会存在服务器作为此客户端在服务器的唯一标识符,与客户端提供的verifier关联在一起

    Network File System, Ops(1): EXCHANGE_ID
        [Program Version: 4]
        [V4 Procedure: COMPOUND (1)]
        Status: NFS4_OK (0)
        Tag: <EMPTY>
            length: 0
            contents: <EMPTY>
        Operations (count: 1)
            Opcode: EXCHANGE_ID (42)
                Status: NFS4_OK (0)
                clientid: 0x5c6a7d06a9000080    
                seqid: 0x00000001
                flags: 0x00020000, EXCHGID4_FLAG_USE_PNFS_MDS
                    0... .... .... .... .... .... .... .... = EXCHGID4_FLAG_CONFIRMED_R: Not set
                    .0.. .... .... .... .... .... .... .... = EXCHGID4_FLAG_UPD_CONFIRMED_REC_A: Not set
                    .... .... .... .0.. .... .... .... .... = EXCHGID4_FLAG_USE_PNFS_DS: Not set
                    .... .... .... ..1. .... .... .... .... = EXCHGID4_FLAG_USE_PNFS_MDS: Set
                    .... .... .... ...0 .... .... .... .... = EXCHGID4_FLAG_USE_NON_PNFS: Not set
                    .... .... .... .... .... ...0 .... .... = EXCHGID4_FLAG_BIND_PRINC_STATEID: Not set
                    .... .... .... .... .... .... .... ..0. = EXCHGID4_FLAG_SUPP_MOVED_MIGR: Not set
                    .... .... .... .... .... .... .... ...0 = EXCHGID4_FLAG_SUPP_MOVED_REFER: Not set
                eia_state_protect: SP4_NONE (0)
                eir_server_owner
                    minor ID: 1
                    major ID: <DATA>
                        length: 44
                        contents: <DATA>
                server scope: <DATA>
                    length: 3
                    contents: <DATA>
                    fill bytes: opaque data
                eir_server_impl_id
                    Implementor DNS domain name(nii_domain): <EMPTY>
                        length: 0
                        contents: <EMPTY>
                    Implementation product name(nii_name): cycle_test_src_0
                        length: 16
                        contents: cycle_test_src_0
                    Build timestamp(nii_date)
                        seconds: 0
                        nseconds: 0
        [Main Opcode: EXCHANGE_ID (42)]
    

    CREATE_SESSION
    这一步就是最重要的客户端发起建立会话请求,通过此请求告知会话中使用的BLOCK SIZE等信息,会话一旦建立就会一直保存到umount或者服务器重启为止:

    Network File System, Ops(1): CREATE_SESSION
        [Program Version: 4]
        [V4 Procedure: COMPOUND (1)]
        Tag: <EMPTY>
            length: 0
            contents: <EMPTY>
        minorversion: 1
        Operations (count: 1): CREATE_SESSION
            Opcode: CREATE_SESSION (43)
                clientid: 0x5c6a7d06a9000080
                seqid: 0x00000001
                csa_flags: 0x00000003, CREATE_SESSION4_FLAG_PERSIST, CREATE_SESSION4_FLAG_CONN_BACK_CHAN
                    .... .... .... .... .... .... .... ...1 = CREATE_SESSION4_FLAG_PERSIST: Set
                    .... .... .... .... .... .... .... ..1. = CREATE_SESSION4_FLAG_CONN_BACK_CHAN: Set
                    .... .... .... .... .... .... .... .0.. = CREATE_SESSION4_FLAG_CONN_RDMA: Not set
                csa_fore_chan_attrs
                    hdr pad size: 0
                    max req size: 1049620
                    max resp size: 1049480
                    max resp size cached: 3416
                    max ops: 8
                    max reqs: 64
                csa_back_chan_attrs
                    hdr pad size: 0
                    max req size: 4096
                    max resp size: 4096
                    max resp size cached: 0
                    max ops: 2
                    max reqs: 1
                cb_program: 0x40000000
                flavor: 1
                stamp: 693003078
                machine name: drmcyc-dengh2-1-00.cec.lab.emc.com
                    length: 34
                    contents: drmcyc-dengh2-1-00.cec.lab.emc.com
                    fill bytes: opaque data
                uid: 0
                gid: 0
        [Main Opcode: CREATE_SESSION (43)]
    
    

    服务器回复,其中最重要的便是Session ID:04000000067d6a5c7856341204000000

    Network File System, Ops(1): CREATE_SESSION
        [Program Version: 4]
        [V4 Procedure: COMPOUND (1)]
        Status: NFS4_OK (0)
        Tag: <EMPTY>
            length: 0
            contents: <EMPTY>
        Operations (count: 1)
            Opcode: CREATE_SESSION (43)
                Status: NFS4_OK (0)
                sessionid: 04000000067d6a5c7856341204000000
                seqid: 0x00000001
                csr_flags: 0x00000002, CREATE_SESSION4_FLAG_CONN_BACK_CHAN
                    .... .... .... .... .... .... .... ...0 = CREATE_SESSION4_FLAG_PERSIST: Not set
                    .... .... .... .... .... .... .... ..1. = CREATE_SESSION4_FLAG_CONN_BACK_CHAN: Set
                    .... .... .... .... .... .... .... .0.. = CREATE_SESSION4_FLAG_CONN_RDMA: Not set
                csr_fore_chan_attrs
                    hdr pad size: 0
                    max req size: 1049620
                    max resp size: 1049480
                    max resp size cached: 3416
                    max ops: 8
                    max reqs: 64
                csr_back_chan_attrs
                    hdr pad size: 0
                    max req size: 4096
                    max resp size: 4096
                    max resp size cached: 0
                    max ops: 2
                    max reqs: 1
        [Main Opcode: CREATE_SESSION (43)]
    

    RECLAIM_COMPLETE
    这一步先忽略,以后在高可靠性的文章里再详细的解释,目前忽略掉不影响我们理解。

    SECINFO_NO_NAME
    客户将不使用任何认证用户访问NFS共享目录。

    PUTROOTFH
    获取NFS根目录信息,客户端会保存这些信息以便随后的文件访问。

    相关文章

      网友评论

          本文标题:万事有商量,论沟通的重要性 (NFS篇)

          本文链接:https://www.haomeiwen.com/subject/ghasoqtx.html