前言

  在学了py的静态爬虫之后，总是感觉有点缺什么，仔细一想，静态的网页会了，但是有的网站人家用的是ajax搞的，这可怎么办啊！于是乎，就想搞点动态爬虫玩玩。

---

开搞

1.安装requests和json

2.伪装浏览器

url = 'https://book.qidian.com/ajax/book/category?_csrfToken=BXnzDKmnJamNAgLu4O3GknYVL2YuNX5EE86tTBAm&bookId=3144877'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36',
               'Referer': 'https://book.qidian.com/info/3144877',
               'Cookie': '_csrfToken=BXnzDKmnJamNAgLu4O3GknYVL2YuNX5EE86tTBAm; newstatisticUUID=1564467217_1193332262; qdrs=0%7C3%7C0%7C0%7C1; showSectionCommentGuide=1; qdgd=1; lrbc=1013637116%7C436231358%7C0%2C1003541158%7C309402995%7C0; rcr=1013637116%2C1003541158; bc=1003541158%2C1013637116; e1=%7B%22pid%22%3A%22qd_P_limitfree%22%2C%22eid%22%3A%22qd_E01%22%2C%22l1%22%3A4%7D; e2=%7B%22pid%22%3A%22qd_P_free%22%2C%22eid%22%3A%22qd_A18%22%2C%22l1%22%3A3%7D'
               }

3.使用requests发送请求 ，并转码

res = requests.get(url=url, params=headers)
s = str(res.content, "utf-8")

4.使用json进行str到json的转换

load = json.loads(s)

打印结果

5.逐层解析

    for (k, v) in load.items():  # 类型为dict
        if k == "data":
            for (i, j) in v.items():  # 类型为dict
                if i == "vs":
                    for p in j:  # 类型为list
                        for (a, b) in p.items():  # 类型为dict
                            if a == "vId" or a == "cCnt" or a == "vN":  # py自带的判断
                                print(a, b)
                            if a == "cs":
                                for q in b:
                                    print(q)

打印结果

6.这些只是用于解析非vip的书，VIP的咱还不会啊