美文网首页大数据 爬虫Python AI Sql
python 爬虫必备工具箱之----cookies参数一键转换

python 爬虫必备工具箱之----cookies参数一键转换

作者: 盖码范 | 来源:发表于2019-07-22 11:13 被阅读1次

    爬虫就是模拟客服端发送请求,如果网页需要登陆,cookies是模拟发送请求的重要参数之一。不过我们用工具抓到的cookies不是字典类型,如下图。

    抓包cookie

    平常都需要手动一条一条的添加到我们的代码中,接下来介绍一种方法能将网络抓取到的cookies一键转换成字典形式的cookies

    cookie = '''__yadk_uid=6HEhZYxkVNAcAQpBxwlB6GDh98u5VO6M; locale=zh-CN; read_mode=day; default_font=font2; Hm_lvt_0c0e9d9b1e7d617b3e6842e85b9fb068=1563266849,1563333736,1563341800,1563448719; remember_user_token=W1sxNDUzMDM2NF0sIiQyYSQxMSRNL3BibnRpNTFPWkJZcllFbWVudW4uIiwiMTU2Mzc1NzgzNS4zODMyMTc2Il0%3D--8d19a3bd106f8430e6e392469ac5e095f77fb16c; _m7e_session_core=c0679824bb7208a7b3dc095ae52bf9fd; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%2214530364%22%2C%22%24device_id%22%3A%2216a87d305a578e-0827a5662b2f0a-6353160-2073600-16a87d305a6b82%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E7%9B%B4%E6%8E%A5%E6%B5%81%E9%87%8F%22%2C%22%24latest_referrer%22%3A%22%22%2C%22%24latest_referrer_host%22%3A%22%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC_%E7%9B%B4%E6%8E%A5%E6%89%93%E5%BC%80%22%2C%22%24latest_utm_source%22%3A%22recommendation%22%2C%22%24latest_utm_medium%22%3A%22seo_notes%22%2C%22%24latest_utm_campaign%22%3A%22maleskine%22%2C%22%24latest_utm_content%22%3A%22note%22%7D%2C%22first_id%22%3A%2216a87d305a578e-0827a5662b2f0a-6353160-2073600-16a87d305a6b82%22%7D; Hm_lpvt_0c0e9d9b1e7d617b3e6842e85b9fb068=1563760114'''
    
    {c.split('=')[0].replace(' ', ''): c.split('=')[1] for c in cookie.split(';')}
    Out[10]: 
    {'__yadk_uid': '6HEhZYxkVNAcAQpBxwlB6GDh98u5VO6M',
     'locale': 'zh-CN',
     'read_mode': 'day',
     'default_font': 'font2',
     'Hm_lvt_0c0e9d9b1e7d617b3e6842e85b9fb068': '1563266849,1563333736,1563341800,1563448719',
     'remember_user_token': 'W1sxNDUzMDM2NF0sIiQyYSQxMSRNL3BibnRpNTFPWkJZcllFbWVudW4uIiwiMTU2Mzc1NzgzNS4zODMyMTc2Il0%3D--8d19a3bd106f8430e6e392469ac5e095f77fb16c',
     '_m7e_session_core': 'c0679824bb7208a7b3dc095ae52bf9fd',
     'sensorsdata2015jssdkcross': '%7B%22distinct_id%22%3A%2214530364%22%2C%22%24device_id%22%3A%2216a87d305a578e-0827a5662b2f0a-6353160-2073600-16a87d305a6b82%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E7%9B%B4%E6%8E%A5%E6%B5%81%E9%87%8F%22%2C%22%24latest_referrer%22%3A%22%22%2C%22%24latest_referrer_host%22%3A%22%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC_%E7%9B%B4%E6%8E%A5%E6%89%93%E5%BC%80%22%2C%22%24latest_utm_source%22%3A%22recommendation%22%2C%22%24latest_utm_medium%22%3A%22seo_notes%22%2C%22%24latest_utm_campaign%22%3A%22maleskine%22%2C%22%24latest_utm_content%22%3A%22note%22%7D%2C%22first_id%22%3A%2216a87d305a578e-0827a5662b2f0a-6353160-2073600-16a87d305a6b82%22%7D',
     'Hm_lpvt_0c0e9d9b1e7d617b3e6842e85b9fb068': '1563760114'}
    
    

    代码

    {c.split('=')[0].replace(' ', ''): c.split('=')[1] for c in cookie.split(';')}
    

    相关文章

      网友评论

        本文标题:python 爬虫必备工具箱之----cookies参数一键转换

        本文链接:https://www.haomeiwen.com/subject/kpjilctx.html