相关疑难解决方法(0)

我正在寻找一种快速保存大型numpy数组的方法.我想以二进制格式将它们保存到磁盘,然后相对快速地将它们读回内存.不幸的是,cPickle还不够快.

我找到了numpy.savez和numpy.load.但奇怪的是,numpy.load将npy文件加载到"memory-map"中.这意味着定期操作数组确实很慢.例如,像这样的东西会非常慢:

#!/usr/bin/python
import numpy as np;
import time; 
from tempfile import TemporaryFile

n = 10000000;

a = np.arange(n)
b = np.arange(n) * 10
c = np.arange(n) * -0.5

file = TemporaryFile()
np.savez(file,a = a, b = b, c = c);

file.seek(0)
t = time.time()
z = np.load(file)
print "loading time = ", time.time() - t

t = time.time()
aa = z['a']
bb = z['b']
cc = z['c']
print "assigning time = ", time.time() - t; …