连接 CNN 来比较两个图像

Question

我正在训练 CNN 来比较两个图像。然后 CNN 可以判断我放入 CNN 中的新图像是否相等。因此，我通过 cv2 连接图像，以便获得形状为 (330,530,6)（rgb 形式）的图像，或者就像我以灰度（330,530,2）进行操作一样。 这对我来说效果很好，但我想知道如果我在展平两个模型后连接两个模型，CNN 是如何工作的。我正在使用 keras 序列模型。有没有办法在不改变所有内容的情况下连接这两个模型？因为我在 fit 方法中给出了数据，所以我不确定如何将这两个数据移交给每个模型。

这是我使用的模型：

CNN = Sequential()
CNN.add(layers.Conv2D(32,(3,3),activation='relu',kernel_regularizer=regularizers.l2(l2Reg),input_shape=(330,530,2)))
CNN.add(layers.MaxPool2D(pool_size=(2, 2)))
CNN.add(layers.Conv2D(32,(3,3),activation='relu',kernel_regularizer=regularizers.l2(l2Reg)))
CNN.add(layers.MaxPool2D(pool_size=(3, 3)))
CNN.add(layers.Conv2D(64,(3,3),activation='relu',kernel_regularizer=regularizers.l2(l2Reg)))
CNN.add(layers.MaxPool2D(pool_size=(3, 3)))
CNN.add(layers.Conv2D(64,(3,3),activation='relu',kernel_regularizer=regularizers.l2(l2Reg)))
CNN.add(layers.MaxPool2D(pool_size=(3, 3)))
CNN.add(layers.Flatten())
CNN.add(layers.Dropout(0.5))
CNN.add(layers.Dense(128,activation='relu',kernel_regularizer=regularizers.l2(l2Reg)))
CNN.add(layers.Dense(2,activation='softmax'))
CNN.summary()
CNN.compile(optimizer=optimizers.RMSprop(lr=1e-4),loss='categorical_crossentropy',metrics=['accuracy'])
history=CNN.fit(XTrain,YTrain,epochs=40,batch_size=32,validation_split=0.15)
scores = CNN.evaluate(XTest,YTest,batch_size=32)
print("Accuracy: %.2f%%" % (scores[1]*100))
CNN.save("AnodenerkennungDreiV")

Answer 1

您可以创建一个输入列表，在您的情况下有 2 个项目，比方说 'image1' 和 'image2'。然后，您可以为每个以 Flatten 层结尾的图像创建一个卷积层和池化层的分支。

330x530 灰度图像示例：

import numpy as np
from keras.utils import plot_model
from keras.models import Model
from keras.layers import Conv2D, MaxPool2D, Flatten, Input, concatenate, Dense

inputs = [Input(shape=(330,530,1), name='image1'), Input(shape=(330,530,1), name='image2')]

flattened_layers = []
for input in inputs:
    conv_layer = Conv2D(32,(3,3),activation='relu')(input)
    conv_layer = MaxPool2D(pool_size=(2,2))(conv_layer) 
    # note that previous layer is used as input for creating the next layer,
    # you'll need to do this for every layer.

    # add more layers here
    flattened_layers.append(Flatten()(conv_layer))

output = concatenate(flattened_layers, axis=0) #you have to check which axis you want to use here
#add more layers here
output = Dense(2,activation='softmax')(output)

model = Model(inputs=inputs, outputs=output)
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])
plot_model(model, "C:/help_me_pls/example_model.png", show_shapes=True)

要将数据输入此模型，您将需要一个字典作为 X 值。

X['image1']

应包含所有第一张图像和

X['image2']

所有第二张图像。您应该仍然能够使用相同的 y 值。

下图显示了示例模型的架构：

我希望我正确理解了你的问题，这就是你正在寻找的。

Answer 2

我需要帮助.. 我正在编写这样的代码，我想检查我是否走在正确的道路上：

比较两组输入图像，这些输入彼此略有不同，让 CNN 学习有什么差异，并稍后将这些差异应用到新插入的未见过的图像中。

import os
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Reshape # Import Reshape here
from tensorflow.keras.preprocessing.image import ImageDataGenerator
import matplotlib.pyplot as plt


train_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
    '/content/drive/MyDrive/Train',
    target_size=(200, 200), 
    batch_size=32,
    color_mode='grayscale',  # Explicitly set color mode to grayscale
    class_mode='input') 

test_datagen = ImageDataGenerator(rescale=1./255)
test_generator = test_datagen.flow_from_directory(
    '/content/drive/MyDrive/Test',
    target_size=(200, 200),
    batch_size=32,
    color_mode='grayscale',  # Explicitly set color mode to grayscale
    class_mode='input')
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='sigmoid', input_shape=(200, 200, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
# Reshape the output to match the input image shape
model.add(Dense(200 * 200 * 1, activation='sigmoid'))
# Use tf.keras.layers.Reshape instead of just Reshape
model.add(tf.keras.layers.Reshape((200, 200, 1))) # Reshape the output to match the input image shape
model.compile(optimizer='adam', loss='mse', metrics=['mse'])
history = model.fit(
    train_generator,
    epochs=50,  # Adjust number of epochs
    validation_data=test_generator)
# steps_per_epoch and validation_steps are inferred automatically by Keras
# from the length of the generators.
import numpy as np # Import the numpy library
import matplotlib.pyplot as plt
import numpy as np

new_image_path = '/content/drive/MyDrive/Output/output/11.jpg'
# Load the image in grayscale by specifying color_mode='grayscale'
new_image = tf.keras.utils.load_img(new_image_path, target_size=(200, 200), color_mode='grayscale') 
new_image = tf.keras.utils.img_to_array(new_image)
new_image = np.expand_dims(new_image, axis=0)

predicted_image = model.predict(new_image)
# Reshape to (200, 200) by removing the extra dimension
predicted_image = predicted_image.reshape((200, 200))  

# Assuming predicted_image is still grayscale, use cmap='gray'
plt.imshow(predicted_image, cmap='gray')  
plt.show()