pcap2txt

作者: 低级bug制造专家 | 来源:发表于2019-08-22 15:08 被阅读0次

该程序的作用是将pcap文件里的前三个ssl/tls包的数据，转化成十进制提取到txt文件里：

dataset里有很多域名文件夹，每个文件夹下有很多的txt文件，记录着数据包，我们使不同域名文件夹下的txt文件个数相同
pcapnum_per_txt：每个txt文件，是由pcapnum_per_txt个pcap处理得到的
filelist是所有pcap文件名的列表，这些pcap要分txt_num次处理，每次处理的文件名存进new_list里

if __name__ == '__main__':
    path = '/home/new3/https/lx/login.weixin.qq.com'
    filelist = os.listdir(path)
    #print (len(filelist))

    pcapnum_per_txt = len(filelist) // txt_num

    for i in range(txt_num):
        print(str(i)+'.txt')
        new_list = filelist[i*pcapnum_per_txt:i*pcapnum_per_txt + pcapnum_per_txt]


        for file in new_list:
            print('The pcap file is: ' + file)
            filepath = os.path.join(path,file)
            parse_pcap(filepath)

        txt_name = str(i) + '.txt'
        fw = open(txt_name, "a+")
        for key in flow.keys():
            if key in new_list:
                if(len(flow[key]) == 3):
                    print(key)
                    for pkts in flow[key]:
                        for bytes in pkts:
                            fw.write(str(bytes)+" ")
                        fw.write("\n")
        print('The above pcap file is written in the txt file.')
        print ('\n')

parse_pcap

先读取24字节的pcap文件头，然后在循环读【先16字节数据包头包含这个数据包的大小iplensave，再读iplensave大小的数据包】
参数iplensave记录了当前数据包的长度

确定当前包是ssl/tls的方法：

tls是基于tcp的，由tcp封装
tls包的第一个字段标明了tls类型（content type），接下来的一个字段标明了version，大多数（目前发现）的content type 值只有20（0x14），22（0x16），23（0x17）三个值，version的第一个字节都是由0x03开头的

tls.png

mac层有14B，ip层一般有20B，tcp的长度不定长，由首部的header length字段给出了tcp层的长度，该字段只有4bits，位于tcp首部的第13B的前4b，该值转化为十进制再乘4就是整个tcp的长度
首先判断包长度iplensave > 54，因为mac+ip+tcp最少需要54B，小于54B一定没有tls层
判断iplensave - tcplen - iplen - maclen > 0，如果=0也没有tls层
两个条件都满足，再判断tcp的下一字节是不是20，22或者23，version是不是3，两者都满足，则是tls包（巧合的概率很小可以忽略）

tcp

def parse_pcap(filename):
    with open(filename, "rb") as file: 
        # Read 24-bytes pcap header 
        data = file.read(pcaphdrlen)
        (tag, maj, min, tzone, ts, ppsize, lt) = struct.unpack("=L2p2pLLLL", data)
        # pocket counter
        cnt = 0

        while data:
            # read packet header
            data = file.read(pkthdrlen)
            if not data:
                break
            (sec, microsec, iplensave, origlen) = struct.unpack("=LLLL", data)
            # print (sec, microsec, iplensave, origlen)
            #print iplensave
            data = file.read(iplensave)


            if iplensave > 54:
                tcplen = ord(data[46])//16*4
                if iplensave - tcplen - iplen - maclen > 0:
                    tlstype = maclen + iplen + tcplen
                    tlsversion = tlstype + 1
                    if (ord(data[tlstype]) == 20 or ord(data[tlstype]) == 22 or ord(data[tlstype]) == 23) and ord(data[tlsversion]) == 3:
                        processpacket(data)
                        cnt = cnt + 1

        print('The number of ssl/tls packets: ' + str(cnt))
        print('----------------------------------------------------------------------------------')

processpacket

定义一个字典flow，key是pcap的文件名，因为我们输入的是一个域名文件夹下的所有pcap文件，使用文件名作为key不会有重复，value是该pcap文件下满足筛选条件（tls）的包，最多取三个

def processpacket(pkt):
    pkt = [ord(b) for b in str(pkt)]
    proto = pkt[23]

    srcip = "{0}.{1}.{2}.{3}".format(pkt[26], pkt[27], pkt[28], pkt[29])
    dstip = "{0}.{1}.{2}.{3}".format(pkt[30], pkt[31], pkt[32], pkt[33])

    sport = pkt[34] * 256 + pkt[35]
    dport = pkt[36] * 256 + pkt[37]

    pkt = preprocess(pkt, proto)
    # print file    

    tuple = file
    if tuple in flow:

        value = flow[tuple]
        if len(value) < 3:
            value.append(pkt)
            flow[tuple] = value

    else:
        value = []

        value.append(pkt)
        flow[tuple] = value

preprocess

取1000字节，去掉mac和ip层的信息，从tcp开始截取

def preprocess(packet, proto):
    # remove mac and ip layer, start from tcp layer
    packet = packet[34:]
    #TCP
    if len(packet) < 1000:
        for j in range(1000 - len(packet)):
            packet.append(0)
    else:
        packet = packet[:1000]
    return packet

结果：

横线部分是程序处理的pcap文件，ssl/tls包的个数是3的才会被写进txt里，小于3的不处理
横线底下列出的是写进txt的pcap文件，列出的顺序就是写入txt的顺序

new3@new3:~/https/lx$ python parsepcap.py 
0.txt
The pcap file is: 159.226.121.15_54806_101.226.76.164_443_1556368053.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.117.158_44082_101.227.160.102_443_1556357309.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.171.251_5794_101.227.160.102_443_1556370252.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.244_60644_101.227.160.102_443_1556357290.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.253_12310_223.166.152.108_443_1556368139.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.25.81_7116_101.226.76.164_443_1556368107.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.20.7_35292_101.226.76.164_443_1556368849.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.117.215_1098_101.227.160.102_443_1556357592.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
159.226.25.81_7116_101.226.76.164_443_1556368107.pcap
159.226.171.251_5794_101.227.160.102_443_1556370252.pcap
159.226.117.215_1098_101.227.160.102_443_1556357592.pcap
159.226.35.244_60644_101.227.160.102_443_1556357290.pcap
159.226.20.7_35292_101.226.76.164_443_1556368849.pcap
The above pcap file is written in the txt file.


1.txt
The pcap file is: 159.226.171.251_34568_101.226.76.164_443_1556369112.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.121.15_49579_101.226.76.164_443_1556357432.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.171.251_34358_101.227.160.102_443_1556368031.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.113.225_14779_101.226.76.164_443_1556369020.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.118.121_53831_101.226.76.164_443_1556357383.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.171.251_14133_101.227.160.102_443_1556368271.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.182.51_49968_117.135.169.34_443_1556368833.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.25.91_36511_101.226.76.164_443_1556368921.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
159.226.25.91_36511_101.226.76.164_443_1556368921.pcap
159.226.171.251_14133_101.227.160.102_443_1556368271.pcap
159.226.182.51_49968_117.135.169.34_443_1556368833.pcap
159.226.171.251_34568_101.226.76.164_443_1556369112.pcap
159.226.171.251_34358_101.227.160.102_443_1556368031.pcap
The above pcap file is written in the txt file.


2.txt
The pcap file is: 159.226.95.33_12289_101.226.76.164_443_1556370074.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.43.54_48313_101.226.76.164_443_1556357346.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.118.132_54887_101.227.160.102_443_1556357319.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.118.121_58915_101.226.76.164_443_1556368004.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.25.91_32511_101.227.160.102_443_1556367961.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.172_5276_101.227.160.102_443_1556370303.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.113.225_33547_101.226.76.164_443_1556370161.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.118.121_58948_101.226.76.164_443_1556368244.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
159.226.25.91_32511_101.227.160.102_443_1556367961.pcap
159.226.95.33_12289_101.226.76.164_443_1556370074.pcap
159.226.43.54_48313_101.226.76.164_443_1556357346.pcap
159.226.35.172_5276_101.227.160.102_443_1556370303.pcap
159.226.118.121_58948_101.226.76.164_443_1556368244.pcap
159.226.113.225_33547_101.226.76.164_443_1556370161.pcap
The above pcap file is written in the txt file.


3.txt
The pcap file is: 159.226.117.158_7200_101.227.160.102_443_1556369012.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.199.87_60649_101.227.160.102_443_1556368176.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.110.25_65292_101.226.76.164_443_1556368947.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.171.251_1215_101.227.160.102_443_1556357407.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.113.225_55106_101.226.76.164_443_1556357379.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.25.81_7611_101.226.76.164_443_1556368229.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.117.158_23010_101.226.76.164_443_1556370332.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
The pcap file is: 159.226.118.138_51823_101.226.76.164_443_1556370162.pcap
The number of ssl/tls packets: 2
----------------------------------------------------------------------------------
159.226.171.251_1215_101.227.160.102_443_1556357407.pcap
159.226.199.87_60649_101.227.160.102_443_1556368176.pcap
159.226.25.81_7611_101.226.76.164_443_1556368229.pcap
The above pcap file is written in the txt file.


4.txt
The pcap file is: 159.226.35.244_53259_101.226.76.164_443_1556368032.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.21.20_53148_101.227.160.102_443_1556357379.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.244_53897_101.227.160.102_443_1556368813.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.117.158_12623_101.227.160.102_443_1556368952.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.244_55147_101.226.76.164_443_1556370313.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.231.165_52310_101.226.76.164_443_1556367992.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.25.81_6935_101.226.76.164_443_1556368047.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
The pcap file is: 159.226.35.177_55157_101.226.76.164_443_1556357535.pcap
The number of ssl/tls packets: 3
----------------------------------------------------------------------------------
159.226.35.177_55157_101.226.76.164_443_1556357535.pcap
159.226.35.244_55147_101.226.76.164_443_1556370313.pcap
159.226.35.244_53259_101.226.76.164_443_1556368032.pcap
159.226.21.20_53148_101.227.160.102_443_1556357379.pcap
159.226.25.81_6935_101.226.76.164_443_1556368047.pcap
159.226.231.165_52310_101.226.76.164_443_1556367992.pcap
159.226.35.244_53897_101.227.160.102_443_1556368813.pcap
159.226.117.158_12623_101.227.160.102_443_1556368952.pcap
The above pcap file is written in the txt file.


new3@new3:~/https/lx$

pcap2txt

parse_pcap

确定当前包是ssl/tls的方法：

processpacket

preprocess

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读