“文本生成不支持模型‘MPTForCausalLM’”-尝试使用 MPT-7B 指令时出现以下警告

Question

我正在使用 GCP 的虚拟机（e2-highmem-4（高效实例，4 个 vCPU，32 GB RAM））来加载模型并使用它。这是我写的代码-

import torch
from transformers import pipeline
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import transformers
config = transformers.AutoConfig.from_pretrained(
  'mosaicml/mpt-7b-instruct',
  trust_remote_code=True,
)
# config.attn_config['attn_impl'] = 'flash'

model = transformers.AutoModelForCausalLM.from_pretrained(
  'mosaicml/mpt-7b-instruct',
  config=config,
  torch_dtype=torch.bfloat16,
  trust_remote_code=True,
  cache_dir="./cache"
)
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b", cache_dir="./cache")
text_gen = pipeline("text-generation", model=model, tokenizer=tokenizer)
text_gen(text_inputs="what is 2+2?")

Now the code is taking way too long to generate the text. Am I doing something wrong? or is there any way to make things faster? Also, when creating the pipeline, I am getting the following warning-\

The model 'MPTForCausalLM' is not supported for text-generation

I tried generating text by using it but it was stuck for a long time.

Answer 1

您可能想尝试 GPU 实例，因为尝试在 CPU 上使用更大的 LLMS 现在几乎是失败的原因。

不管怎样，我还得到了“文本生成不支持模型‘MPTForCausalLM’”这就是我最终进入这个线程的原因。尽管有警告，文本生成确实对我有用。