美文网首页
python requests pymongo

python requests pymongo

作者: 站长_郭生 | 来源:发表于2017-10-13 19:28 被阅读0次
try:
    import json as simplejson
except:
    import simplejson

from pymongo import MongoClient
import requests


def GetMongoDb():
    mongo = MongoClient('127.0.0.1', 27017)
    db = mongo.filminfo
    return db


url = "http://www.baidu.com"
# res = http.HttpReq("GET", url, {}, {}, 35)
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.104 Safari/537.36 Core/1.53.3427.400 QQBrowser/9.6.12201.400"}
res = requests.get(url, headers=headers)
# print(res)
if res.status_code == 200:
    db = GetMongoDb()
    resp = simplejson.loads(res.content, strict=False)
    for val in resp["urlset"]:
          db.test.update({"id": val["data"]["id"]}, {"$set": val["data"]}, upsert = True)

相关文章

网友评论

      本文标题:python requests pymongo

      本文链接:https://www.haomeiwen.com/subject/rjhtuxtx.html