美文网首页
selenium、chromedirver

selenium、chromedirver

作者: 阿发贝塔伽马 | 来源:发表于2022-10-08 09:31 被阅读0次

    driver = webdriver.Chrome()
    driver.get("https://www.zhipin.com/web/geek/job?query=%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90&city=100010000")
    

    获取当前页面信息,一定要等待网页加载完成,针对反爬措施,要点手动与自动相结合,手动登录(自动难以实现),翻页可以自动,一定要注意到driver始终获取的是当前页面最新信息

    elements = driver.find_elements(By.CSS_SELECTOR, ".job-card-wrapper")
    jobinfo_csv(elements)
    
    def jobTaglist(taglist):
        strTaglist = ""
        for el in taglist:
            strTaglist += el.text+"*"
        return strTaglist
    def jobinfo_csv(elements):
        jobInfo = {}
        for element in elements:
            jobInfo["job-name"] = element.find_element(By.CSS_SELECTOR, ".job-name").text
            jobInfo['job-area'] = element.find_element(By.CSS_SELECTOR, ".job-area").text
            jobInfo['salary'] = element.find_element(By.CSS_SELECTOR, ".salary").text
            t = element.find_element(By.CSS_SELECTOR, ".job-card-footer .tag-list").find_elements(By.XPATH, "./li")
            jobInfo['technologylist']= jobTaglist(t)
            taglist = element.find_element(By.CSS_SELECTOR, ".tag-list").find_elements(By.XPATH, "./li")
            jobInfo['taglist']= jobTaglist(taglist)
            companyTaglist = element.find_element(By.CSS_SELECTOR, ".company-tag-list").find_elements(By.XPATH, "./li")
            jobInfo['companyTaglist']= jobTaglist(companyTaglist)
            jobInfo['company-name'] = element.find_element(By.CSS_SELECTOR, ".company-name").text
            jobInfo['info-desc'] = element.find_element(By.CSS_SELECTOR, ".info-desc").text
            pddata = pd.DataFrame([jobInfo])
            pddata.to_csv('static.csv',mode='a',encoding='utf-8',header=False,index=False)
    

    相关文章

      网友评论

          本文标题:selenium、chromedirver

          本文链接:https://www.haomeiwen.com/subject/ntviartx.html