小编Utr*_*rax的帖子

我正在尝试使用 llama-cpp-python （llama.cpp 的 Python 包装器）使用 Google Colab 中的 Llama LLM 进行推理。我的代码如下所示：

!pip install llama-cpp-python
from llama_cpp import ChatCompletionMessage, Llama

model = Llama(
    "/content/drive/MyDrive/<weights-file>.bin",
)

但是，当运行它时，我收到此错误：

AssertionError                            Traceback (most recent call last)
<ipython-input-13-652eb650093d> in <cell line: 9>()
      7 }
      8 
----> 9 model = Llama(
     10     model_path="/content/drive/MyDrive/careo/Wizard-Vicuna-13B-Uncensored.ggmlv3.q4_1.bin",
     11 )

/usr/local/lib/python3.10/dist-packages/llama_cpp/llama.py in __init__(self, model_path, n_ctx, n_parts, n_gpu_layers, seed, f16_kv, logits_all, vocab_only, use_mmap, use_mlock, embedding, n_threads, n_batch, last_n_tokens_size, lora_base, lora_path, low_vram, tensor_split, rope_freq_base, rope_freq_scale, n_gqa, rms_norm_eps, mul_mat_q, verbose)
    321                     self.model_path.encode("utf-8"), self.params
    322 …