huggingface 下载配置

配置 huggingface 镜像, git clone深度, 启用加速下载包(rust实现)

# huggingface
export HF_ENDPOINT="https://hf-mirror.com"
export GIT_LFS_SKIP_SMUDGE=1
export HF_HUB_ENABLE_HF_TRANSFER=1

下载工具包及加速下载包(启用加速工具 hf-transfer 需要环境变量 HF_HUB_ENABLE_HF_TRANSFER 为 1)

pip install -U huggingface_hub
pip install -U hf-transfer

生成本地 token 文件

from huggingface_hub.hf_api import HfFolder
HfFolder.save_token('hf_your_token_string')

huggingface 会把token写入到: ~/.cache/huggingface/token

import torch; print(torch.cuda.is_available())