如何在pytorch基本示例中包括批量大小？

Question

我是pytorch的新手。以下是使用nn模块使用一些随机数据训练简单的单层模型的基本示例（从此处开始）

import torch
N, D_in, H, D_out = 64, 1000, 100, 10

x = torch.randn(N, D_in)
y = torch.randn(N, D_out)

model = torch.nn.Sequential(
    torch.nn.Linear(D_in, H),
    torch.nn.ReLU(),
    torch.nn.Linear(H, D_out),
)
loss_fn = torch.nn.MSELoss(reduction='sum')
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)
for t in range(500):
    y_pred = model(x)

    loss = loss_fn(y_pred, y)
    print(t, loss.item())

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

据我了解，该示例中的批处理大小等于1，换句话说，一个点（64个点）用于计算梯度和更新参数。我的问题是：如何修改此示例以训练批大小大于1的模型？

Answer 1

要在 PyTorch 基本示例中包含批量大小，最简单、最干净的方法是使用 PyTorchtorch.utils.data.DataLoader和torch.utils.data.TensorDataset.

Dataset 存储样本及其相应的标签，DataLoader 围绕 Dataset 包装一个可迭代对象，以便轻松访问样本。

DataLoader将负责为您创建批次。

基于您的问题，有一个完整的代码片段，其中我们迭代 2 个时期的 10000 个示例的数据集，批量大小为 64：

import torch
from torch.utils.data import DataLoader, TensorDataset


# Create the dataset with N_SAMPLES samples
N_SAMPLES, D_in, H, D_out = 10000, 1000, 100, 10

x = torch.randn(N_SAMPLES, D_in)
y = torch.randn(N_SAMPLES, D_out)

# Define the batch size and the number of epochs
BATCH_SIZE = 64
N_EPOCHS = 2

# Use torch.utils.data to create a DataLoader 
# that will take care of creating batches 
dataset = TensorDataset(x, y)
dataloader = DataLoader(dataset, batch_size=BATCH_SIZE, shuffle=True)

# Define model, loss and optimizer
model = torch.nn.Sequential(
    torch.nn.Linear(D_in, H),
    torch.nn.ReLU(),
    torch.nn.Linear(H, D_out),
)

loss_fn = torch.nn.MSELoss(reduction='sum')
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)

# Get the dataset size for printing (it is equal to N_SAMPLES)
dataset_size = len(dataloader.dataset)

# Loop over epochs
for epoch in range(N_EPOCHS):
    print(f"Epoch {epoch + 1}\n-------------------------------")

    # Loop over batches in an epoch using DataLoader
    for id_batch, (x_batch, y_batch) in enumerate(dataloader):

        y_batch_pred = model(x_batch)

        loss = loss_fn(y_batch_pred, y_batch)

        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        # Every 100 batches, print the loss for this batch
        # as well as the number of examples processed so far 
        if id_batch % 100 == 0:
            loss, current = loss.item(), (id_batch + 1)* len(x_batch)
            print(f"loss: {loss:>7f}  [{current:>5d}/{dataset_size:>5d}]")

输出应该类似于：

Epoch 1
-------------------------------
loss: 643.433716  [   64/10000]
loss: 648.195435  [ 6464/10000]
Epoch 2
-------------------------------
loss: 613.619873  [   64/10000]
loss: 625.018555  [ 6464/10000]

Answer 2

实际上N是批量大小。所以你只需要将N当前它的设置修改为 64。所以你在每个训练批次中都有 64 个大小为 / dim 的向量D_in。

我检查了您发布的链接，您也可以查看评论-也有一些解释:)

# -*- coding: utf-8 -*-
import numpy as np

# N is batch size; D_in is input dimension;
# H is hidden dimension; D_out is output dimension.
N, D_in, H, D_out = 64, 1000, 100, 10

# Create random input and output data
x = np.random.randn(N, D_in)
y = np.random.randn(N, D_out)

# Randomly initialize weights
w1 = np.random.randn(D_in, H)
w2 = np.random.randn(H, D_out)

learning_rate = 1e-6
for t in range(500):
    # Forward pass: compute predicted y
    h = x.dot(w1)
    h_relu = np.maximum(h, 0)
    y_pred = h_relu.dot(w2)

    # Compute and print loss
    loss = np.square(y_pred - y).sum()
    print(t, loss)

    # Backprop to compute gradients of w1 and w2 with respect to loss
    grad_y_pred = 2.0 * (y_pred - y)
    grad_w2 = h_relu.T.dot(grad_y_pred)
    grad_h_relu = grad_y_pred.dot(w2.T)
    grad_h = grad_h_relu.copy()
    grad_h[h < 0] = 0
    grad_w1 = x.T.dot(grad_h)

    # Update weights
    w1 -= learning_rate * grad_w1
    w2 -= learning_rate * grad_w2