使用目标数据中的缺失值训练 LSTM 神经网络 - 错误 optim.step()

Question

我想用目标数据中的缺失值和用户定义的损失函数来训练 LSTM 神经网络。然而，optim.step()之后出现错误，一些权重/偏差是nan。这方面有什么提示吗？谢谢

import torch
import numpy as np
from torch import nn

# Define a simple lstm model
class myLSTM(nn.Module):
    def __init__(self, input_size, hidden_size):
        super(myLSTM, self).__init__()
        self.lstm = nn.LSTM(input_size, hidden_size)
    def forward(self, input):
        output, _ = self.lstm(input)
        return output

# Input and target
input = torch.randn(10, 5, requires_grad=True)
target = torch.randn(10, 5)

# There is one missing values in the target data
target[0,0] = np.nan

# Create model
lstmModel = myLSTM(5, 5) 

# Loss function, optimization
def loss_function(y_true, y_predict):
    return torch.nanmean((y_true-y_predict)**2)

optim = torch.optim.Adam(lstmModel.parameters(), lr=0.01)

# Training with only 1 epoch
output = lstmModel(input)
optim.zero_grad()
error = loss_function(target, output)
error.backward()
optim.step()

lstmModel.state_dict()

Answer 1

一种解决方案是屏蔽

nan

元素，请尝试以下操作：

def loss_function(y_true, y_predict):
    mask = ~torch.isnan(y_true)
    return torch.mean((y_true[mask] - y_predict[mask])**2)