组合两个预训练模型的输出（在不同数据集上训练）并使用某种形式的二元分类器来预测图像

Question

我有两个预训练模型。

Model_1 = Inception Model with Imagenet Dataset (1000 classes)

My_Model = Inception Model trained with a custom dataset (20 classes)通过转移学习和微调

我想将两个模型（Model_1和My_Model）的输出结合在一个新层中。

新图层应使用一些二元分类器来判断是否使用Model_1或My_Model进行基于输入图像的预测。

例如：

如果我试图预测“狗”图像，结合两个模型的二元分类器应该说我需要使用Model_1来预测狗图像（因为My_Model数据集没有用Dog图像训练）而Model_1是用Dog图像训练的。

谁能告诉我如何实现这一目标？一些示例实现或代码片段将有所帮助。

谢谢

Answer 1

为此，您需要制作组合模型，然后在另一个自定义数据集上训练组合模型，这是组合模型的外观示例。要制作数据集，只需获取每个图像并确定您要使用的模型，然后您可以训练组合模型的输出，以便为一个模型提供正值，为另一个模型提供负值。希望能帮助到你

import numpy as np
import pandas as pd
import keras
from keras.layers import Dense, Flatten, Concatenate
from tensorflow.python.client import device_lib
# check for my gpu 
print(device_lib.list_local_devices())


# making some models like the ones you have
input_shape = (10000, 3)
m1_input = Input(shape = input_shape, name = "m1_input")
fc = Flatten()(m1_input)
m1_output = Dense(1000, activation='sigmoid',name = "m1_output")(fc)
Model_1 = Model(m1_input,m1_output)

m2_input = Input(shape = input_shape, name = "m2_input")
fc = Flatten()(m2_input)
m2_output = Dense(20, activation='sigmoid',name = "m2_output")(fc)
My_Model = Model(m2_input,m2_output)


# set the trained models to be untrainable
for layer in Model_1.layers:
    layer.trainable = False
for layer in My_Model.layers:
    layer.trainable = False

#build a combined model
combined_model_input = Input(shape = input_shape, name = "combined_model_input")
m1_predict = Model_1(combined_model_input)
m2_predict = My_Model(combined_model_input)
combined = Concatenate()([m1_predict, m2_predict])
fc = Dense(500, activation='sigmoid',name = "fc1")(combined)
fc = Dense(100, activation='sigmoid',name = "fc2")(fc)
output_layer = Dense(1, activation='tanh',name = "fc3")(fc)
model = Model(combined_model_input, output_layer)

#check the number of parameters that are trainable
print(model.summary())

#psudocode to show how to make a training set for the combined model:

    combined_model_y= []
    for im in images:
        if class_of(im) in list_of_my_model_classes:
            combined_model_y.append(1)
        else:
            combined_model_y.append(-1)
    combined_model_y = np.array(combined_model_y)

# then train the combined model:
model.compile('adam', loss = 'binary_crossentropy')
model.fit(images, combined_model_y, ....)