爬取网页的通用代码框架
作者:
帅气的_xiang | 来源:发表于
2017-04-18 20:47 被阅读48次import requests
def getHTMLText(url):
try:
headers = {
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36'
}
r = requests.get(url, headers=headers, timeout=30)
r.raise_for_status() #如果状态不是200，引发HTTPError异常
r.encoding = r.apparent_encoding
return r.text
except:
return "产生异常"
if __name__ == '__main__':
url = "http://www.baidu.com"
print(getHTMLText(url))
本文标题：爬取网页的通用代码框架
本文链接：https://www.haomeiwen.com/subject/ftpqzttx.html
网友评论