2020-11-19

import requests

from lxml import etree

for i in range(9):

        url = 'http://xiaohua.zol.com.cn/youmo/{}.html'.format(i)

        headers = {'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36'}

        resp = requests.get(url,headers=headers)    

        resp_ = etree.HTML(resp.text)

        resp_xpath = resp_.xpath("//div[@class='summary-text']/text()")

        for i in resp_xpath:

                content = '\n'+i

                with open('./xiaohuapa.txt','a',encoding='utf-8') as f:

                        f.write(content)

爬取小说，content = '\n'+i，很好用可以换行