python爬虫基础笔记

安装anaconda

百度anaconda 清华镜像

python下载

python下载网址

pycharm下载

pycharm下载网址

response\request

import requests
headers={'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.87 Safari/537.36'}
response = requests.get('https://www.baidu.com',headers=headers)
# print(response.content)
print(response.status_code)
# print(response.text)
print(response.headers)

结果为

200
{'Transfer-Encoding': 'chunked', 'Strict-Transport-Security': 'max-age=172800', 'Date': 'Wed, 19 Apr 2017 02:24:38 GMT', 'X-UA-Compatible': 'IE=Edge,chrome=1', 'Content-Type': 'text/html; charset=utf-8', 'Set-Cookie': 'BAIDUID=4C38A6525791B3B6807F40F3EE3A9EFE:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com, BIDUPSID=4C38A6525791B3B6807F40F3EE3A9EFE; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com, PSTM=1492568678; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com, BDSVRTM=0; path=/, BD_HOME=0; path=/, H_PS_PSSID=22584_1464_21108_22176_20928; path=/; domain=.baidu.com, __bsi=18299889646158639724_00_34_N_N_3_0303_C02F_N_N_N_0; expires=Wed, 19-Apr-17 02:24:43 GMT; domain=www.baidu.com; path=/', 'Vary': 'Accept-Encoding', 'Cache-Control': 'private', 'Cxy_all': 'baidu+c53eebf51fcba6b6f9e98964a052e732', 'Connection': 'keep-alive', 'BDQID': '0x8c88176e00002334', 'P3P': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Content-Encoding': 'gzip', 'BDPAGETYPE': '1', 'Expires': 'Wed, 19 Apr 2017 02:24:29 GMT', 'Server': 'bfe/1.0.8.18', 'X-Powered-By': 'HPHP', 'BDUSERID': '0'}

抓取图片并写入文件

import requests
response = requests.get('https://ss1.bdstatic.com/70cFuXSh_Q1YnxGkpoWK1HF6hhy/it/u=611483611,2895064642&fm=117&gp=0.jpg')
# print(response.content)
with open('/1.gif','wb') as f:
    f.write(response.content)
    f.close()

下载库的网址

下载库的网址