接口自动化测试---通用JSON解析算法

作者: Qi仔 | 来源:发表于2017-08-28 18:00 被阅读83次

JSON格式的数据在http接口的自动化测试中是一种常见的数据结构，在接口自动化脚本里解析JSON返回结果,验证数据的正确性是非常关键的一步，但JSON结构有简单有复杂，不同的接口返回不同的结果，自动化脚本中解析响应JSON数据占了一大部分的工作量，而且随着接口的变动，维护脚本也特别麻烦，今天在这里介绍一种通用的解析所有JSON 的通用算法，来解决这一问题。

算法解析：

首先来看一段较为复杂的JSON:

图-1

从图-1可以看出，这段JSON第一层包含了resultCode，resultDesc，data，dataList四个key值，其中resultCode，resultDesc，data 都是单个的字符串，而dataList 的值是一个列表，有俩元素，其中每个元素又是一个字典，包括id,name,caption,subjectId,notes,items,其中items中又是一个list,每个元素为一个字典，key值为 id,name,caption等....不难发现这段JSON中第二层和第三层中都包含相同的key: id，name, capiton,这也是选择这段JSON的原因。大多数的JSON中可能都会有相同的key,解析的时候如何精确的获取想要的key对应的值这也是需要考虑的场景之一。

其实JSON的整个结构就是 “树”，可以将上述JSON中的第一个节点[JSON]看成树的根节点， resultCode，resultDesc 等节点看成叶子节点，而dataList 则可以看成父节点，dataList里的元素可以看成是子节点。

来一张更容易理解的图：A 为根节点，黄色的节点称是父节点，绿色节点则是叶子节点，图中的D 类比为图-1中的dataList节点，B,C为图一种的resultCode，resultDesc

图-2

解析JSON主要分为以下几步

第一步：

将JSON 转化为树，从根节点（A）开始遍历每一路径，即从根节点到叶子节点每一条路径作为一条数据，并且获取每一深度（JSON中的每一层）叶子节点的{key:value}，其中父节点的值不取（通常父节点可能是一个list,或者一个字典）,当前深度的叶子节点的值全取，与下一深度的叶子节点的值加到同一字典中。
如图-2，B,C,D,E是深度为2的节点，第一次遍历，第一条数据为{B,C},以此递归，直至遍历所有路径，可到图-2的遍历结果为(其中B节点理解为 {'B':'B'},图-3 为图-2的JSON格式)

图-3

按第一步解析结果如下：

1. [{'B': 'B'}, {'C': 'C'}]

2. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}]

3. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'K': 'K'}, {'L': 'L'}, {'M': 'M'}]

4. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}]

5. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}, {'S': 'S'}, {'R': 'R'}]

6. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}, {'S': 'S'}, {'R': 'R'}, {'T': 'T'}]

7. [{'B': 'B'}, {'C': 'C'}, {'U': 'U'}]

分析上述结果可以看出第3组结果包含了第1,2组值，6包含了5,4的结果，因此可以将重复的数据进行一次过滤和筛选

第二步：

过滤重复数据，步骤一的结果可以过滤为：

3. [{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'K': 'K'}, {'L': 'L'}, {'M': 'M'}]

6.[{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}, {'O': 'O'}, {'P': 'P'}, {'S': 'S'}, {'R': 'R'}, {'T': 'T'}]

7. [{'B': 'B'}, {'C': 'C'}, {'U': 'U'}]

在这一步骤中，如果不同深度的叶子节点名称重复，则按照深度，依次在key后面加入序号1,2...

即如果有3个B,则解析结果为：

[{'B': 'B'}, {'B1': 'B1'}, {'B2: 'B2'}, {'F': 'F'}, {'K': 'K'}, {'L': 'L'}, {'M': 'M'}]

第三步：

根据想要获取的KEY,在第二步获取到的list中取出对应的value，KEY以列表的方式给出

keys = [key1,key2,key3....]，如要获取[B,C,F,G]

则最终结果为：

[{'B': 'B'}, {'C': 'C'}, {'G': 'G'}, {'F': 'F'}]

这样就轻而易举的解析处理我们想要的结果!

代码 -python算法实现

def responseComplexJson2LD(responseJson,assertKeys):
    '''
    :param responseJson: 类型：JSONObject 接口返回:JSON 格式
    :param assertKeys: 类型：list 需要校验的key ['resultCode','resultDesc','id','name','caption','id1','notes','name1','caption1','cubeName','functions']
                       说明：如果key里有重复的解析为 key1,key2...
    :return: list[dict]
    '''
    try:
        if not isinstance(responseJson,dict):
            raise Exception('responseJson 类型错误，必须为JSON格式')

        result_list = []

        def isInclude(dict1, dict2):
            '''
            :function: 判断 dict1 是否包含于dict2
            :param dict1:
            :param dict2:
            :return: True False
            '''
            for key1 in dict1:
                if key1 in dict2:
                    pass
                else:
                    return False
            return True

        def parse_dict(responseJson, parent):
            '''
            :递归解析json,将json树解析成list[dict,dict....]
            :param responseJson:
            :param parent:
            :return:
            '''
            if isinstance(responseJson, dict):
                dp = parent[:]
                not_l_d = [responseKey for responseKey in responseJson if not isinstance(responseJson[responseKey], (dict, list))]
                for i in not_l_d:
                    dp.append({i: responseJson[i]})
                dp_1 = dp[:]
                result_list.append(dp)
                for responseKey in responseJson:
                    if isinstance(responseJson[responseKey], (dict, list)):
                        parse_dict(responseJson[responseKey], dp_1)
            elif isinstance(responseJson, list):
                for i in responseJson:
                    dd = parent[:]
                    if not isinstance(i, (dict, list)):
                        dd.append(i)
                        result_list.append(dd)
                    else:
                        parse_dict(i, dd)
            else:
                dx = parent[:]
                dx.append(responseJson)
                result_list.append(dx)


        parse_dict(responseJson, parent=[])
        if not result_list:
            return []
        '''获取给定的assertKeys'''
        last_result =[]
        templist ={}
        if assertKeys:
            if isinstance(assertKeys,list):
                    for iter in result_list:
                        for key in iter:
                            thiskey = list(key.keys())[0]
                            '''如果有重复的key,解析为key1,key2...'''
                            i = 1
                            temp = thiskey
                            while True:
                                if thiskey in list(templist.keys()):
                                    thiskey = thiskey + str(i)
                                    i = i + 1
                                else:
                                    templist.setdefault(thiskey,key.get(temp))
                                    break
                        last_result.append(templist)
                        templist={}
            else:
                raise Exception("assertKeys 类型错误，必须为list")

        '''将assertKeys里的定义的key加入dict,不在assertKeys 定义的除去'''
        assert_result = []
        templist = {}
        for meta in last_result:
            keys = list(meta.keys())
            if isInclude(assertKeys,keys):
                for key in assertKeys:
                    templist.setdefault(key,meta.get(key))
                    assert_result.append(templist)
                templist={}
            else:
                pass

        '''对返回的last_result中的元素去重'''
        result = []
        for meta in assert_result:
            if meta not in result:
                result.append(meta)
            else:
                pass
        return result
    except Exception as err:
        Log.error(err)
        return []

结果验证

以图-1中的json 为例测试

测试1：

keylist = ['resultCode']

responseComplexJson2LD(test,keylist)

返回：

{'resultCode': 100}

测试2：

keylist = ['resultCode','resultDesc']

responseComplexJson2LD(test,keylist)

返回：

{'resultCode': 100, 'resultDesc': '成功'}

测试3：

keylist = ['resultCode','resultDesc','id']

responseComplexJson2LD(test,keylist)

返回：

{'resultCode': 100, 'resultDesc': '成功', 'id': 1}

{'resultCode': 100, 'resultDesc': '成功', 'id': 2}

测试4：--重复id,第二用id1标识

keylist = ['resultCode', 'resultDesc', 'id','id1']

responseComplexJson2LD(test,keylist)

返回：

{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 1}

{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 2}

{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 20}

{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 3}

{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 4}

{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 5}

{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 6}

{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 21}

{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 22}

{'resultCode': 100, 'resultDesc': '成功', 'id': 1, 'id1': 23}

{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 7}

{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 8}

{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 9}

{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 10}

{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 24}

{'resultCode': 100, 'resultDesc': '成功', 'id': 2, 'id1': 25}

测试5：取第二个id

keylist = ['resultCode', 'resultDesc','id1']

responseComplexJson2LD(test,keylist)

返回：

{'resultCode': 100, 'resultDesc': '成功', 'id1': 1}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 2}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 20}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 3}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 4}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 5}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 6}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 21}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 22}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 23}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 7}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 8}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 9}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 10}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 24}

{'resultCode': 100, 'resultDesc': '成功', 'id1': 25}

测试6：只取caption 字段，俩caption 时第二个写caption1

keylist = ['caption','caption1]

responseComplexJson2LD(test,keylist)

返回：

{'caption': '交易', 'caption1': '交易金额'}

{'caption': '交易', 'caption1': '交易用户数'}

{'caption': '交易', 'caption1': '人均交易金额'}

{'caption': '交易', 'caption1': '订单数'}

{'caption': '交易', 'caption1': '订单金额'}

{'caption': '交易', 'caption1': '大订单数'}

{'caption': '交易', 'caption1': '大订单金额'}

{'caption': '交易', 'caption1': '平均订单金额'}

{'caption': '交易', 'caption1': '大订单数占比'}

{'caption': '交易', 'caption1': '平均大订单金额'}

{'caption': '盈亏', 'caption1': '盈利用户数'}

{'caption': '盈亏', 'caption1': '亏损用户数'}

{'caption': '盈亏', 'caption1': '净盈利值'}

{'caption': '盈亏', 'caption1': '净亏损值'}

{'caption': '盈亏', 'caption1': '盈利用户数占比'}

{'caption': '盈亏', 'caption1': '亏损用户数占比'}

附上图-1完整的Json报文：

data = {
    "resultCode": 100,
    "resultDesc": "成功",
    "data": "",
    "dataList": [
        {
            "id": 1,
            "subjectId": "",
            "name": "trading",
            "caption": "交易",
            "notes": "1.人均交易金额=用户累积交易金额/交易用户数<br>2.平均订单金额=用户累计订单金额/订单数<br>3.大订单：单笔手续费大于50元的订单（手续费＝每笔订单金额＊万分之八）<br>4.大订单数占比=大订单数/订单数<br>5.平均大订单金额=累计大订单金额/大订单数<br>",
            "items": [
                {
                    "id": 1,
                    "name": "pay_money",
                    "caption": "交易金额",
                    "cubeName": "TRANSACTION_ANALYSIS_V3",
                    "columns": "CONTQTY",
                    "itemType": "SINGLE",
                    "calType": "",
                    "valCalType": "MONEY",
                    "functions": "SUM"
                },
                {
                    "id": 2,
                    "name": "pay_user_num",
                    "caption": "交易用户数",
                    "cubeName": "TRANSACTION_ANALYSIS_V3",
                    "columns": "ACCOUNT_ID",
                    "itemType": "SINGLE",
                    "calType": "",
                    "valCalType": "COMMON",
                    "functions": "COUNT_DISTINCT"
                },
                {
                    "id": 20,
                    "name": "pay_money,pay_user_num",
                    "caption": "人均交易金额",
                    "cubeName": "TRANSACTION_ANALYSIS_V3",
                    "columns": "CONTQTY,ACCOUNT_ID",
                    "itemType": "COMBINE",
                    "calType": "/",
                    "valCalType": "MONEY",
                    "functions": "SUM,COUNT_DISTINCT"
                },
                {
                    "id": 3,
                    "name": "order_num",
                    "caption": "订单数",
                    "cubeName": "TRANSACTION_ANALYSIS_V3",
                    "columns": "*",
                    "itemType": "SINGLE",
                    "calType": "",
                    "valCalType": "COMMON",
                    "functions": "COUNT"
                },
                {
                    "id": 4,
                    "name": "order_money",
                    "caption": "订单金额",
                    "cubeName": "TRANSACTION_ANALYSIS_V3",
                    "columns": "CONTQTY",
                    "itemType": "SINGLE",
                    "calType": "",
                    "valCalType": "MONEY",
                    "functions": "SUM"
                },
                {
                    "id": 5,
                    "name": "big_order_num",
                    "caption": "大订单数",
                    "cubeName": "TRANSACTION_ANALYSIS_V3",
                    "columns": "*",
                    "itemType": "CONDITION",
                    "calType": "",
                    "valCalType": "COMMON",
                    "functions": "COUNT"
                },
                {
                    "id": 6,
                    "name": "big_order_money",
                    "caption": "大订单金额",
                    "cubeName": "TRANSACTION_ANALYSIS_V3",
                    "columns": "CONTQTY",
                    "itemType": "CONDITION",
                    "calType": "",
                    "valCalType": "MONEY",
                    "functions": "SUM"
                },
                {
                    "id": 21,
                    "name": "order_money,order_num",
                    "caption": "平均订单金额",
                    "cubeName": "TRANSACTION_ANALYSIS_V3",
                    "columns": "CONTQTY,*",
                    "itemType": "COMBINE_CONDITION",
                    "calType": "/",
                    "valCalType": "MONEY",
                    "functions": "SUM,COUNT"
                },
                {
                    "id": 22,
                    "name": "big_order_num,order_num",
                    "caption": "大订单数占比",
                    "cubeName": "TRANSACTION_ANALYSIS_V3",
                    "columns": "*,*",
                    "itemType": "COMBINE_CONDITION",
                    "calType": "/",
                    "valCalType": "PERCENT",
                    "functions": "COUNT,COUNT"
                },
                {
                    "id": 23,
                    "name": "big_order_money,big_order_num",
                    "caption": "平均大订单金额",
                    "cubeName": "TRANSACTION_ANALYSIS_V3",
                    "columns": "CONTQTY,*",
                    "itemType": "COMBINE_CONDITION",
                    "calType": "/",
                    "valCalType": "MONEY",
                    "functions": "SUM,COUNT"
                }
            ]
        },
        {
            "id": 2,
            "subjectId": "",
            "name": "profit&loss",
            "caption": "盈亏",
            "notes": "<br>1.盈利用户数：当日用户中，净利润&gt;0的用户数<br>2.亏损用户数：当日用户中，净利润&lt;0的用户数<br>3.盈利用户占比=当日盈利用户数/当日总用户数<br>4.亏损用户占比=当日亏损用户数/当日总用户数<br>5.净盈利值：当日用户中，净利润&gt;0的值求和<br>6.净亏损值：当日用户中，净利润&lt;0的值求和<br>7.日净利润＝当日净资产-前一日净资产-当日入金+当日出金<br>8.月净利润=月末净资产-月初净资产-月内总入金+月内总出金<br>其他时间粒度计算方法类似月粒度<br>",
            "items": [
                {
                    "id": 7,
                    "name": "profit_user_count",
                    "caption": "盈利用户数",
                    "cubeName": "NET_PROFIT_ANALYSIS_V3",
                    "columns": "ACCOUNT_ID",
                    "itemType": "CONDITION",
                    "calType": "",
                    "valCalType": "COMMON",
                    "functions": "COUNT_DISTINCT"
                },
                {

                    "id": 8,
                    "name": "loss_user_count",
                    "caption": "亏损用户数",
                    "cubeName": "NET_PROFIT_ANALYSIS_V3",
                    "columns": "ACCOUNT_ID",
                    "itemType": "CONDITION",
                    "calType": "",
                    "valCalType": "COMMON",
                    "functions": "COUNT_DISTINCT"
                },
                {
                    "id": 9,
                    "name": "net_profit_num",
                    "caption": "净盈利值",
                    "cubeName": "NET_PROFIT_ANALYSIS_V3",
                    "columns": "NETPROFIT",
                    "itemType": "CONDITION",
                    "calType": "",
                    "valCalType": "MONEY",
                    "functions": "SUM"
                },
                {
                    "id": 10,
                    "name": "net_loss_num",
                    "caption": "净亏损值",
                    "cubeName": "NET_PROFIT_ANALYSIS_V3",
                    "columns": "NETPROFIT",
                    "itemType": "CONDITION",
                    "calType": "",
                    "valCalType": "MONEY",
                    "functions": "SUM"
                },
                {
                    "id": 24,
                    "name": "profit_user_count,profit_total_user_count",
                    "caption": "盈利用户数占比",
                    "cubeName": "NET_PROFIT_ANALYSIS_V3",
                    "columns": "ACCOUNT_ID,ACCOUNT_ID",
                    "itemType": "COMBINE_CONDITION",
                    "calType": "/",
                    "valCalType": "PERCENT",
                    "functions": "COUNT_DISTINCT,COUNT_DISTINCT"
                },
                {
                    "id": 25,
                    "name": "loss_user_count,profit_total_user_count",
                    "caption": "亏损用户数占比",
                    "cubeName": "NET_PROFIT_ANALYSIS_V3",
                    "columns": "ACCOUNT_ID,ACCOUNT_ID",
                    "itemType": "COMBINE_CONDITION",
                    "calType": "/",
                    "valCalType": "PERCENT",
                    "functions": "COUNT_DISTINCT,COUNT_DISTINCT"
                }
            ]
        }
    ]
    }

接口自动化测试---通用JSON解析算法

算法解析：

解析JSON主要分为以下几步

第一步：

按第一步解析结果如下：

第二步：

第三步：

代码 -python算法实现

结果验证

测试1：

测试2：

测试3：

测试4：--重复id,第二用id1标识

测试5：取第二个id

测试6：只取caption 字段，俩caption 时第二个写caption1

附上图-1完整的Json报文：

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

QAHome