大家好，我是小张！
在《Python制作词云视频，通过词云图来看小姐姐跳舞》文章中简单介绍了B站弹幕的爬取方法，只需找到视频中的参数 cid，就能采集到该视频下的所有弹幕；思路虽然很简单，但个人感觉还是比较麻烦，例如之后的某一天，我想采集B站上的某个视频弹幕，还需要从头开始：找cid参数、写代码，重复单调；
因此我在想有没有可能一步到位，以后采集某个视频弹幕时只需一步操作，比如输入想爬取的视频链接，程序能自动识别下载
实现效果
基于此，借助 PyQt5 我写了一个小工具，只需提供目标视频的 url 以及目标 txt 路径，程序对该视频下的弹幕自动采集并把数据保存至目标txt文本，先看一下预览效果：
效果
PS 微信公号对动图帧数有限制，制作动图时我删减了一部分内容，因此效果可能会不太流畅
工具实现整体分为UI界面、数据采集 两个部分，用到的Python库：
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n290" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">import requests
import re
from PyQt5.QtWidgets import *
from PyQt5 import QtCore
from PyQt5.QtGui import *
from PyQt5.QtCore import QThread, pyqtSignal
from bs4 import BeautifulSoup</pre>
UI界面
UI 界面借助了 PyQt5，放置了了两个按钮(开始下载、保存至)，输入视频链接 的 editline 控件及调试窗口；
image-20210217185155009
代码如下：
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n267" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"> def init(self,parent =None):
super(Ui_From,self).init(parent=parent)
self.setWindowTitle("B站弹幕采集")
self.setWindowIcon(QIcon('pic.jpg'))# 图标
self.top_label = QLabel("作者：小张\n 微信公号：小张Python")
self.top_label.setAlignment(QtCore.Qt.AlignHCenter)
self.top_label.setStyleSheet('color:red;font-weight:bold;')
self.label = QLabel("B站视频url")
self.label.setAlignment(QtCore.Qt.AlignHCenter)
self.editline1 = QLineEdit()
self.pushButton = QPushButton("开始下载")
self.pushButton.setEnabled(False)#关闭启动
self.Console = QListWidget()
self.saveButton = QPushButton("保存至")
self.layout = QGridLayout()
self.layout.addWidget(self.top_label,0,0,1,2)
self.layout.addWidget(self.label,1,0)
self.layout.addWidget(self.editline1,1,1)
self.layout.addWidget(self.pushButton,2,0)
self.layout.addWidget(self.saveButton,3,0)
self.layout.addWidget(self.Console,2,1,3,1)
self.setLayout(self.layout)
self.savepath = None
self.pushButton.clicked.connect(self.downButton)
self.saveButton.clicked.connect(self.savePushbutton)
self.editline1.textChanged.connect(self.syns_lineEdit)</pre>
当 url 不为空以及目标文本存放路径已经设置好之后，才能进入数据采集模块
效果12
实现此功能的代码：
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n270" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;"> def syns_lineEdit(self):
if self.editline1.text():
self.pushButton.setEnabled(True)#打开按钮
def savePushbutton(self):
savePath = QFileDialog.getSaveFileName(self,'Save Path','/','txt(*.txt)')
if savePath[0]:# 选中 txt 文件路径
self.savepath = str(savePath[0])#进行赋值</pre>
数据采集
程序获取到 url 之后，第一步就是访问 url 提取当前页面中该视频的cid 参数(一连串数字)
image-20210217194745469
利用cid 参数构造该存放该视频弹幕的 API 接口，随后用常规 requests 和 bs4 包实现文本采集
image-20210217195252765
数据采集部分代码：
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n277" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">f = open(self.savepath, 'w+', encoding='utf-8') # 打开 txt 文件
res = requests.get(url)
res.encoding = 'utf-8'
soup = BeautifulSoup(res.text, 'lxml')
items = soup.find_all('d') # 找到 d 标签
for item in items:
text = item.text
f.write(text)
f.write('\n')
f.close()</pre>
cid 参数 并不是位于常规 html 的标签上，提取时我选择 re 正则匹配；但这一步骤比较耗机子内存，为了减少对UI界面响应速度的影响，这一步骤单独用一个线程来实现
<pre spellcheck="false" class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" cid="n279" mdtype="fences" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">class Parsetext(QThread):
trigger = pyqtSignal(str) # 信号发射；
def init(self,text,parent = None):
super(Parsetext,self).init()
self.text = text
def del(self):
self.wait()
def run(self):
print('解析 -----------{}'.format(self.text))
result_url = re.findall('.?"baseUrl":"(.?)","base_url".*?', self.text)[0]
self.trigger.emit(result_url)</pre>
小结
好了，以上就是本篇文章的全部内容了，希望内容能够对你工作或者学习上有所帮助。
最后感谢大家的阅读，我们下期见~
源码获取
关于本篇文章用到的源码，可关注微信公号 小张Python，后台回复关键字 210217 即可获取！
网友评论