美文网首页
python requests pymongo

python requests pymongo

作者: 站长_郭生 | 来源:发表于2017-10-13 19:28 被阅读0次
    try:
        import json as simplejson
    except:
        import simplejson
    
    from pymongo import MongoClient
    import requests
    
    
    def GetMongoDb():
        mongo = MongoClient('127.0.0.1', 27017)
        db = mongo.filminfo
        return db
    
    
    url = "http://www.baidu.com"
    # res = http.HttpReq("GET", url, {}, {}, 35)
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.104 Safari/537.36 Core/1.53.3427.400 QQBrowser/9.6.12201.400"}
    res = requests.get(url, headers=headers)
    # print(res)
    if res.status_code == 200:
        db = GetMongoDb()
        resp = simplejson.loads(res.content, strict=False)
        for val in resp["urlset"]:
              db.test.update({"id": val["data"]["id"]}, {"$set": val["data"]}, upsert = True)
    

    相关文章

      网友评论

          本文标题:python requests pymongo

          本文链接:https://www.haomeiwen.com/subject/rjhtuxtx.html