每次Pytorch正向通行证都会发生变化？

Question

我正在学习pytorch并运行玩具回归问题。我似乎每次对模型运行张量时，预测都会改变，这一事实令我感到困惑。显然不是这样，但是我想念的是什么？

Pytorch版本： 0.4.0

我在没有GPU的情况下运行以消除该潜在问题。

码：

from __future__ import division
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import torch
import torch.utils.data as utils_data
from torch.autograd import Variable
from torch import optim, nn
from torch.utils.data import Dataset 
import torch.nn.functional as F
from torch.nn.init import xavier_uniform_, xavier_normal_,uniform_
from sklearn.datasets import load_boston
from sklearn.metrics import mean_squared_error

cuda=False #set to true uses GPU


#load boston data from scikit
boston = load_boston()
x=boston.data
y=boston.target
y=y.reshape(y.shape[0],1)

#change to tensors
x = torch.from_numpy(x)
y = torch.from_numpy(y)

#create dataset and use data loader
training_samples = utils_data.TensorDataset(x, y)
data_loader = utils_data.DataLoader(training_samples, batch_size=64)

#simple model
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()

        #all the layers
        self.fc1   = nn.Linear(x.shape[1], 20)
        xavier_uniform_(self.fc1.weight.data) #this is how you can change the weight init
        self.drop = nn.Dropout(p=0.5)
        self.fc2   = nn.Linear(20, 1)


    def forward(self, x):
        x = F.relu(self.fc1(x))
        x=  self.drop(x)
        x = self.fc2(x)
        return x

net=Net()

if cuda:
    net.cuda()

# create a stochastic gradient descent optimizer
optimizer = optim.Adam(net.parameters())
# create a loss function (mse)
loss = nn.MSELoss(size_average=True)

# run the main training loop
epochs =50
hold_loss=[]

for epoch in range(epochs):
    cum_loss=0.
    for batch_idx, (data, target) in enumerate(data_loader):
        tr_x, tr_y = data.float(), target.float()
        if cuda:
            tr_x, tr_y = tr_x.cuda(), tr_y.cuda() 

        # Reset gradient
        optimizer.zero_grad()

        # Forward pass
        fx = net(tr_x)
        output = loss(fx, tr_y) #loss for this batch

        cum_loss += output.item() #accumulate the loss

        # Backward 
        output.backward()

        # Update parameters based on backprop
        optimizer.step()
    hold_loss.append(cum_loss/len(training_samples))  

#training loss
plt.plot(np.array(hold_loss))

如果重新运行，这部分每次都会返回不同的预测，实际值不会改变，因此数据的顺序不会改变！

#score the training set
for batch_idx, (data, target) in enumerate(data_loader):
        tr_x, tr_y = data.float(), target.float()
        if batch_idx ==0:
           hold_pred=net(tr_x).data.numpy()
           hold_actual=tr_y.data.numpy().reshape(tr_y.data.numpy().shape[0],1)
        else:
            hold_pred =np.row_stack([hold_pred,net(tr_x).data.numpy()])
hold_actual=np.row_stack([hold_actual,tr_y.data.numpy().reshape(tr_y.data.numpy().shape[0],1)])

#view the first few predictions      
print(hold_pred[0:10]) 
print(hold_actual[0:10]) 

Answer 1

您的网络具有一个Dropout层，该层的目的是随机（p=0.5在此处具有概率）对训练期间（net.train()在推理之前设置的）接收到的数据进行采样。有关更多信息（用法，目的），请参阅文档。

该层可以在测试期间短路（net.eval()在推理之前设置）。