baichuan-7B

版本

运行模型：baichuan-7B

torch==1.13.1
transformers==4.28.1
cuda: 11.4

另一个可跑的服务器的各个库版本：

torch==1.13.1
transformers==4.28.1
cuda: 11.7

只有cuda有点出入

Code

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "../models/baichuan-7b/"

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto", trust_remote_code=True)
inputs = tokenizer('你好！', return_tensors='pt')
outputs = model(**inputs)
print(outputs)

Log

  File "/root/miniconda3/lib/python3.11/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling `cublasCreate(handle)`
../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [0,0,0], thread: [32,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [0,0,0], thread: [33,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [0,0,0], thread: [34,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: [0,0,0], thread: [35,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.

RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasCreate(handle)

这个错误查了一些解答

1. batch size过大
   这个GPU的占用情况应该是够用的

2. shape不对应
   这里的报错，input.shape和self.weight.shape应该是对应的