相关疑难解决方法(0)

Transformers v4.x:将慢速分词器转换为快速分词器

我正在关注变压器的预训练模型xlm-roberta-large-xnli示例

from transformers import pipeline
classifier = pipeline("zero-shot-classification",
                      model="joeddav/xlm-roberta-large-xnli")
Run Code Online (Sandbox Code Playgroud)

我收到以下错误

ValueError: Couldn't instantiate the backend tokenizer from one of: (1) a `tokenizers` library serialization file, (2) a slow tokenizer instance to convert or (3) an equivalent slow tokenizer class to instantiate and convert. You need to have sentencepiece installed to convert a slow tokenizer to a fast one.
Run Code Online (Sandbox Code Playgroud)

我用的是变形金刚版 '4.1.1'

python nlp huggingface-transformers huggingface-tokenizers

9
推荐指数
3
解决办法
4379
查看次数