使用 torch.load() 时我的检查点文件有问题

Question

当我torch.load用来加载一个检查点时：

torch.load('./latest_net_G.pth', map_location='cpu')

我收到运行时错误：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/opt/conda/lib/python3.7/tarfile.py in nti(s)
    186             s = nts(s, "ascii", "strict")
--> 187             n = int(s.strip() or "0", 8)
    188         except ValueError:

ValueError: invalid literal for int() with base 8: '_v2\nq\x03(('

During handling of the above exception, another exception occurred:

InvalidHeaderError                        Traceback (most recent call last)
/opt/conda/lib/python3.7/tarfile.py in next(self)
   2288             try:
-> 2289                 tarinfo = self.tarinfo.fromtarfile(self)
   2290             except EOFHeaderError as e:

/opt/conda/lib/python3.7/tarfile.py in fromtarfile(cls, tarfile)
   1094         buf = tarfile.fileobj.read(BLOCKSIZE)
-> 1095         obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors)
   1096         obj.offset = tarfile.fileobj.tell() - BLOCKSIZE

/opt/conda/lib/python3.7/tarfile.py in frombuf(cls, buf, encoding, errors)
   1036 
-> 1037         chksum = nti(buf[148:156])
   1038         if chksum not in calc_chksums(buf):

/opt/conda/lib/python3.7/tarfile.py in nti(s)
    188         except ValueError:
--> 189             raise InvalidHeaderError("invalid header")
    190     return n

InvalidHeaderError: invalid header

During handling of the above exception, another exception occurred:

ReadError                                 Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/torch/serialization.py in _load(f, map_location, pickle_module, **pickle_load_args)
    555         try:
--> 556             return legacy_load(f)
    557         except tarfile.TarError:

/opt/conda/lib/python3.7/site-packages/torch/serialization.py in legacy_load(f)
    466 
--> 467         with closing(tarfile.open(fileobj=f, mode='r:', format=tarfile.PAX_FORMAT)) as tar, \
    468                 mkdtemp() as tmpdir:

/opt/conda/lib/python3.7/tarfile.py in open(cls, name, mode, fileobj, bufsize, **kwargs)
   1590                 raise CompressionError("unknown compression type %r" % comptype)
-> 1591             return func(name, filemode, fileobj, **kwargs)
   1592 

/opt/conda/lib/python3.7/tarfile.py in taropen(cls, name, mode, fileobj, **kwargs)
   1620             raise ValueError("mode must be 'r', 'a', 'w' or 'x'")
-> 1621         return cls(name, mode, fileobj, **kwargs)
   1622 

/opt/conda/lib/python3.7/tarfile.py in __init__(self, name, mode, fileobj, format, tarinfo, dereference, ignore_zeros, encoding, errors, pax_headers, debug, errorlevel, copybufsize)
   1483                 self.firstmember = None
-> 1484                 self.firstmember = self.next()
   1485 

/opt/conda/lib/python3.7/tarfile.py in next(self)
   2300                 elif self.offset == 0:
-> 2301                     raise ReadError(str(e))
   2302             except EmptyHeaderError:

ReadError: invalid header

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
<ipython-input-15-2abbf3aab3ae> in <module>
----> 1 torch.load('multi_task/checkpoints/latest_pet/latest_net_G.pth.tar', map_location='cpu')

/opt/conda/lib/python3.7/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
    385         f = f.open('rb')
    386     try:
--> 387         return _load(f, map_location, pickle_module, **pickle_load_args)
    388     finally:
    389         if new_fd:

/opt/conda/lib/python3.7/site-packages/torch/serialization.py in _load(f, map_location, pickle_module, **pickle_load_args)
    558             if zipfile.is_zipfile(f):
    559                 # .zip is used for torch.jit.save and will throw an un-pickling error here
--> 560                 raise RuntimeError("{} is a zip archive (did you mean to use torch.jit.load()?)".format(f.name))
    561             # if not a tarfile, reset file offset and proceed
    562             f.seek(0)

RuntimeError: multi_task/checkpoints/latest_pet/latest_net_G.pth.tar is a zip archive (did you mean to use torch.jit.load()?)

这是我保存模型的方法：

    def save_networks(self, epoch):
        """Save all the networks to the disk.

        Parameters:
            epoch (int) -- current epoch; used in the file name '%s_net_%s.pth' % (epoch, name)
        """
        for name in self.model_names:
            if isinstance(name, str):
                save_filename = '%s_net_%s.pth' % (epoch, name)
                save_path = os.path.join(self.save_dir, save_filename)
                net = getattr(self, 'net' + name)

                if len(self.gpu_ids) > 0 and torch.cuda.is_available():
                    if name == 'Rgr':
                        torch.save(net.state_dict(), save_path)
                    else:
                        torch.save(net.module.cpu().state_dict(), save_path)
                        net.cuda(self.gpu_ids[0])
                else:
                    if name == 'Rgr':
                        torch.save(net.state_dict(), save_path)
                    else:
                        torch.save(net.cpu().state_dict(), save_path)

我不知道我的检查点文件有什么问题。因为我实际上可以成功加载我的其他检查点文件。另外，我的 pytorch 版本是1.1.0. 你能帮我解决这个问题吗？

谢谢你。

Answer 1

我找到了解决办法。因为我使用不同的集群来训练和调试， torch每个的版本都不一样。保存模型时，torch版本为1.6.0，加载时为1.1.0。