LSTM 中的分类返回相同的分类值

Question

我有10000个数据，每个数据都有0和1的标签。我想使用LSTM进行分类，因为这是时间序列数据。

input_dim = 1
hidden_dim = 32
num_layers = 2
output_dim = 1


# Here we define our model as a class
class LSTM(nn.Module):
  def __init__(self, input_dim, hidden_dim, num_layers, output_dim):
    super(LSTM, self).__init__()
    self.hidden_dim = hidden_dim
    self.num_layers = num_layers
    self.lstm = nn.LSTM(input_dim, hidden_dim, num_layers, batch_first=True)
    self.fc = nn.Linear(hidden_dim, output_dim)

  def forward(self, x):
    #Initialize hidden layer and cell state
    h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_dim).requires_grad_()
    c0 = torch.zeros(self.num_layers, x.size(0), self.hidden_dim).requires_grad_()

    # We need to detach as we are doing truncated backpropagation through time (BPTT)
    # If we don't, we'll backprop all the way to the start even after going through another batch
    out, (hn, cn) = self.lstm(x, (h0.detach(), c0.detach()))


    # Index hidden state of last time step
    # out.size() --> 100, 32, 100
    # out[:, -1, :] --> 100, 100 --> just want last time step hidden states! 
    out = self.fc(out[:, -1, :])

    # For binomial Classification
    m = torch.sigmoid(out)

    return m

model = LSTM(input_dim=input_dim, hidden_dim=hidden_dim, output_dim=output_dim, num_layers=num_layers)
loss = nn.BCELoss()

optimiser = torch.optim.Adam(model.parameters(), lr=0.00001, weight_decay=0.00006)

num_epochs = 100

# Number of steps to unroll
seq_dim =look_back-1  

for t in range(num_epochs):
    y_train_class = model(x_train)

    output = loss(y_train_class, y_train)

    # Zero out gradient, else they will accumulate between epochs
    optimiser.zero_grad(set_to_none=True)

    # Backward pass
    output.backward()

    # Update parameters
    optimiser.step()

这是结果的示例

此代码最初来自kaggle，我对其进行了编辑以进行分类。拜托，你能告诉我我做错了什么吗？

编辑1： 添加数据加载器 从 torch.utils.data 导入 DataLoader 从 torch.utils.data 导入 TensorDataset

x_train = torch.from_numpy(x_train).type(torch.Tensor)
y_train = torch.from_numpy(y_train).type(torch.Tensor)
x_test = torch.from_numpy(x_test).type(torch.Tensor)
y_test = torch.from_numpy(y_test).type(torch.Tensor)

train_dataloader = DataLoader(TensorDataset(x_train, y_train), batch_size=128, shuffle=True)
test_dataloader = DataLoader(TensorDataset(x_test, y_test), batch_size=128, shuffle=True)

我意识到在检查结果之前我忘记了反转变换。当我这样做时，我从分类中得到了不同的值，但是所有值的范围都是 0.001-0.009，所以当我对它们进行舍入时，结果是相同的。所有分类都标记为 0。

Answer 1

神经网络训练中的一个常见现象是，它们最初会收敛到一个非常简单的问题解决方案，即输出一个恒定的预测，以最小化训练数据的误差。我的猜测是，在您的训练数据中，

0

和

1

类别之间的比率接近于

0.5423

。根据您的模型是否具有足够的复杂性，当给予更多学习步骤时，它可能会学会根据输入做出更具体的预测。

虽然增加纪元数量可能会有所帮助，但您可以对当前的设置做一些更好的事情。目前，您每个周期仅执行一个优化器步骤。通常，您需要每批次一个步骤，并以（小）批次（例如 32 个输入）循环处理数据。为此，最好使用DataLoader，您可以在其中定义批量大小，并在纪元循环内循环数据加载器，类似于此示例。