在 Pytorch 中对预训练的 Resnet 模型实现 Dropout

Question

我正在尝试在 Pytorch 中对预训练的 Resnet 模型实现 Dropout，这是我的代码

    feats_list = []
    for key, value in model._modules.items():
        feats_list.append(value)

    for feat in feats_list:
        if isinstance(feat, nn.Conv2d) or isinstance(feat, nn.Conv1d):
            feats_list.append(nn.Dropout(p=0.5, inplace=True))

    model.features = nn.Sequential(*feats_list)
    print(model.features)

我认为应该对所有 conv2 和 conv1 层应用 dropout，但实际上，只有最后一个 AdaptiveAvgPool2d 附加了 dropout 率。

这就是我得到的

...
  (8): AdaptiveAvgPool2d(output_size=(1, 1))
  (9): Linear(in_features=2048, out_features=1000, bias=True)
  (10): Dropout(p=0.5, inplace=True)

有人可以帮助我吗？谢谢你

这是部分代码块仅供参考

def generic_classifier(model, criterion, optimizer, num_epochs=25):
    # try to select specific layers to freeze or unfreeze from the pretrained model
    # true:trainable;  false: freeze, untraibale
    '''
    n = 0
    for param in model.parameters():
        if n  < 7:
            param.requires_grad = True
        else:
            param.requires_grad = False
        n +=1
    '''
    feats_list = []
    for key, value in model._modules.items():
        feats_list.append(value)

    for feat in feats_list:
        if isinstance(feat, nn.Conv2d) or isinstance(feat, nn.Conv1d):
            feats_list.append(nn.Dropout(p=0.5, inplace=True))
            #print(feat)
    #print(feats_list)

    # modify convolution layers
    model.features = nn.Sequential(*feats_list)
    print(model.features)
    #for name, param in model.named_parameters():
    #    print(name, param.requires_grad)

    # remove all the fully connected layers
    model.fc = nn.Sequential()

    # add a number of fully connected layers of our choice right after the convolutional layers
    model.fc = nn.Sequential(
        # need to know the last layer of selected model architecture
        # resnet50:2048, resnet18: 512, resnet34:512
        # did not find a way to automate this part yet.
        nn.Linear(512, 256),
        nn.ReLU(),
        nn.Linear(256, 256),
        nn.ReLU(),
        nn.Linear(256, 3)
    )

    model = model.to(device)

    # train the model
    model_with_pretrained_train_acc = []
    model_with_pretrained_test_acc = []
    start = time.time()

...
...
        return model

Answer 1

代码中的问题是您将 nn.Dropout 层附加到迭代模型模块的循环中的 feats_list 中。但是，当您将 nn.Dropout 层附加到 feats_list 时，它会立即添加到列表的末尾，导致循环继续对其进行迭代。因此，您在架构的末尾添加了大量的 dropout 层。

以下代码包含一个循环，该循环遍历预训练网络的所有层，如果遇到卷积层，它会创建一个完全相同的卷积层并将其附加到列表中，然后是一个 dropout 层，否则它会按原样附加该层添加丢失层。

model = models.resnet18(pretrained=True)

feats_list = []
for key, value in model.named_children():  
    if isinstance(value, nn.Conv2d) or isinstance(value, nn.Conv1d):
        feats_list.append(nn.Conv2d(
            in_channels=value.in_channels,
            out_channels=value.out_channels,
            kernel_size=value.kernel_size,
            stride=value.stride,
            padding=value.padding,
            bias=value.bias,
        ))
        feats_list.append(nn.Dropout(p=0.5, inplace=True))
    else:
        feats_list.append(value)

# Create a new model with the modified layers
model = nn.Sequential(*feats_list)