利用chrome下载整站资源

1. 用chrome打开网站，按F12快捷键打开开发者工具，再按F5刷新，然后在网络右键以cURL（bash）格式复制所有内容，最后将内容保存到 down.sh 文件中。

保存的内容类似这样：

curl 'https://www.baidu.com/' \
  -H 'Connection: keep-alive' \
  -H 'Pragma: no-cache' \
  -H 'Cache-Control: no-cache' \
  -H 'sec-ch-ua: "Chromium";v="21", " Not;A Brand";v="99"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Windows"' \
  -H 'Upgrade-Insecure-Requests: 1' \
  -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36' \
  -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
  -H 'Sec-Fetch-Site: none' \
  -H 'Sec-Fetch-Mode: navigate' \
  -H 'Sec-Fetch-User: ?1' \
  -H 'Sec-Fetch-Dest: document' \
  -H 'Accept-Language: zh-CN,zh;q=0.9' \
  --compressed 
curl 'https://dss0.bdstatic.com/5aV1bjqh_Q23odCf/static/superman/img/topnav/newfanyi-da0cea8f7e.png' \
  -H 'authority: dss0.bdstatic.com' \
  -H 'pragma: no-cache' \
  -H 'cache-control: no-cache' \
  -H 'sec-ch-ua: "Chromium";v="21", " Not;A Brand";v="99"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36' \
  -H 'sec-ch-ua-platform: "Windows"' \
  -H 'accept: image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8' \
  -H 'sec-fetch-site: cross-site' \
  -H 'sec-fetch-mode: no-cors' \
  -H 'sec-fetch-dest: image' \
  -H 'referer: https://www.baidu.com/' \
  -H 'accept-language: zh-CN,zh;q=0.9' \
  --compressed 

2. 将\\\r\n替换为空来将每个curl变成一行，方便下一步正则表达式处理。

3. 将curl 'http[s]?://([^'?]+)(\?.+)?.+--compressed替换为mkdir -p "\$\(dirname $1\)" & $0 -o "\$\(pwd\)"/$1，这样之后每个curl命令就会自动创建下载目录后并自动下载。

4. 在git bash中使用sh down.sh开始下载。
   下载结果