我有一些非常大的输入张量,在构建它们时遇到了内存问题,所以我将它们一一读入文件中.pt。当我运行生成并保存文件的脚本时,文件变得越来越大,因此我假设张量正确保存。这是该代码:
with open(a_sync_save, "ab") as f:
print("saved")
torch.save(torch.unsqueeze(torch.cat(tensors, dim=0), dim=0), f)
Run Code Online (Sandbox Code Playgroud)
我想一次从文件中读取一定数量的张量,因为我不想再次遇到内存问题。当我尝试读取保存到文件中的每个张量时,我只能设法获取第一个张量。
with open(a_sync_save, "rb") as f:
for tensor in torch.load(f):
print(tensor.shape)
Run Code Online (Sandbox Code Playgroud)
这里的输出是第一个张量的形状,然后平静退出。
我有一个 pandas df ,其中一些列是其中包含数据的列表,我想对列表中的标签进行编码。
我收到此错误:
ValueError: Expected 2D array, got 1D array instead:
from sklearn.preprocessing import OneHotEncoder
mins = pd.read_csv('recipes.csv')
enc = OneHotEncoder(handle_unknown='ignore')
X = mins['Ingredients']
'''
[[lettuce, tomatoes, ginger, vodka, tomatoes]
[lettuce, tomatoes, flour, vodka, tomatoes]
...
[flour, tomatoes, vodka, vodka, mustard]]
'''
enc.fit(X)
Run Code Online (Sandbox Code Playgroud)
我希望得到一列包含正确编码信息的列表
[[lettuce, tomatoes, ginger, vodka, tomatoes]
[lettuce, tomatoes, flour, vodka, tomatoes]
...
[flour, tomatoes, vodka, vodka, mustard]
[[0, 1, 2, 3, 1]
[0, 1, 4, 3, 1]
...
[4, 1, 3, 3, 9]]
Run Code Online (Sandbox Code Playgroud) In keras using tensorflow 1 I could ModelCheckpoint(filepath) and the saved file was a called filepath and then I could call model = load_model(filepath) to load the saved model.
Now the equivalent in tensorflow 2 the ModelCheckpoint creates a directory called filepath and when I follow the documentation here to load the saved model I have to create an empty model then call model.load_weights(filepath).
Here is my callback and fit:
filepath = "filepath"
checkpoint = tf.keras.callbacks.ModelCheckpoint(filepath=filepath, mode='max', monitor='val_accuracy', verbose=2, …Run Code Online (Sandbox Code Playgroud) python-3.x ×3
dataframe ×1
keras ×1
python ×1
pytorch ×1
scikit-learn ×1
tensorflow ×1
tf.keras ×1
torch ×1