保存并重新加载拥抱面微调变压器

Question

我正在尝试重新加载经过微调的 DistilBertForTokenClassification 模型。我正在使用转换器 3.4.0 和 pytorch 版本 1.6.0+cu101。使用 Trainer 训练下载的模型后，我使用 trainer.save_model() 保存模型，并在故障排除中通过 model.save_pretrained() 保存在不同的目录中。我正在使用 Google Colab 并将模型保存到我的 Google 驱动器。在测试模型后，我还在我的测试中评估了模型并获得了很好的结果，但是，当我返回笔记本（或工厂重新启动 colab 笔记本）并尝试重新加载模型时，预测很糟糕。检查目录后，config.json 文件和 pytorch_mode.bin 文件都在那里。下面是完整的代码。

from transformers import DistilBertForTokenClassification

# load the pretrained model from huggingface
#model = DistilBertForTokenClassification.from_pretrained('distilbert-base-cased', num_labels=len(uniq_labels))
model = DistilBertForTokenClassification.from_pretrained('distilbert-base-uncased', num_labels=len(uniq_labels)) 

model.to('cuda');

from transformers import Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir = model_dir +  'mitmovie_pt_distilbert_uncased/results',          # output directory
    #overwrite_output_dir = True,
    evaluation_strategy='epoch',
    num_train_epochs=3,              # total number of training epochs
    per_device_train_batch_size=16,  # batch size per device during training
    per_device_eval_batch_size=64,   # batch size for evaluation
    warmup_steps=500,                # number of warmup steps for learning rate scheduler
    weight_decay=0.01,               # strength of weight decay
    logging_dir = model_dir +  'mitmovie_pt_distilbert_uncased/logs',            # directory for storing logs
    logging_steps=10,
    load_best_model_at_end = True
)

trainer = Trainer(
    model = model,                         # the instantiated  Transformers model to be trained
    args = training_args,                  # training arguments, defined above
    train_dataset = train_dataset,         # training dataset
    eval_dataset = test_dataset             # evaluation dataset
)

trainer.train()

trainer.evaluate()

model_dir = '/content/drive/My Drive/Colab Notebooks/models/'
trainer.save_model(model_dir + 'mitmovie_pt_distilbert_uncased/model')

# alternative saving method and folder
model.save_pretrained(model_dir + 'distilbert_testing')

重启后回到笔记本...

from transformers import DistilBertForTokenClassification, DistilBertConfig, AutoModelForTokenClassification

# retreive the saved model 
model = DistilBertForTokenClassification.from_pretrained(model_dir + 'mitmovie_pt_distilbert_uncased/model', 
                                                        local_files_only=True)

model.to('cuda')

现在从任一目录中进行的模型预测都很糟糕，但是，该模型确实有效并输出了我期望的类数，看来实际训练的权重尚未保存或以某种方式未加载。

Answer 1

尝试使用以下代码：

from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForSequenceClassification.from_pretrained(model_path)

Answer 2

您是否尝试加载由培训师保存在文件夹中的模型：

mitmovie_pt_distilbert_uncased/results

Huggingface 训练器将模型直接保存到定义的output_dir。