使用自定义生成器的预测返回不同的数组大小

Question

我有一个接收 3 个输入并给出 1 个输出的模型（现在的模型只是为了具有与我的实际问题相同的层和大小）。

class CNNmodel():
    def __init__(self,
                 height,
                 width,
                 channels):
        self.height = height
        self.width = width
        self.channels = channels

    def build_model(self):
        inputs1 = Input((self.height, self.width, self.channels))
        inputs2 = Input((self.height, self.width, self.channels))
        inputs3 = Input((self.height, self.width, self.channels))
        
        M1M2 = Subtract()([inputs2, inputs1])
        M1M2E1 = Add()([M1M2, inputs3])
        concat = Concatenate(axis=0)([M1M2, inputs3])
        
        x = Conv2D(32, 1, activation='relu')(inputs1)
        
        sr1 = Conv2D(32, 1, activation='relu')(inputs2)
        sr2 = Conv2D(32, 1, activation='relu')(x)
        sr2 = Conv2D(32, 1, activation='relu')(sr2)
        sr2 = Conv2D(32, 1, activation='relu')(concat)

        addition = Add()([sr1, sr2])
       
        f1_input = Input((self.height, self.width, 32))
        f1 = Conv2D(32, 1, activation='relu')(f1_input)
        f2 = Conv2D(32, 1, activation='relu')(f1)
        f3 = Conv2D(1, 1,  activation='relu')(addition)
        
        
        outputs = Conv2D(3, 1)(f3)
        outputs = Conv2D(3, 1)(outputs)
        
        model = Model([inputs1, inputs2, inputs3], outputs)
        return model

我正在使用这个生成器：

class JoinedGen(tf.keras.utils.Sequence):
    def __init__(self,
                 input_gen1,
                 input_gen2,
                 input_gen3,
                 target_gen,
                 batch_size,
                 preds=False,
                 shuffle=False):
        self.gen1 = input_gen1
        self.gen2 = input_gen2
        self.gen3 = input_gen3
        self.target_gen = target_gen
        self.batch_size = batch_size
        self.preds = preds
        self.shuffle = shuffle
        self.on_epoch_end()

        assert len(input_gen1) == len(target_gen)

    def __len__(self):
        return int(np.floor(len(self.gen1) / self.batch_size))

    
    def __getitem__(self, i):
       
        x1 = self.gen1[i]
        x2 = self.gen2[i]
        x3 = self.gen3[i]
        y = self.target_gen[i]
       
        x1 = x1[np.newaxis, :, :, :]
        x2 = x2[np.newaxis, :, :, :]
        x3 = x3[np.newaxis, :, :, :]
        y = y[np.newaxis, :, :, :]
        
        if self.preds:
            return [x1, x2, x3]
        return [x1, x2, x3], y

    def on_epoch_end(self):
        self.indices = np.arange(len(self.gen1))
        if self.shuffle:
            np.random.shuffle(self.indices)

为了预测，我用

preds=True

来称呼它。

我的数据是：

import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Input, Conv2D, Add, Concatenate, Subtract
from tensorflow.keras.models import Model

x1 = np.random.random((64, 16, 16, 3))
x2 = np.random.random((64, 16, 16, 3))
x3 = np.random.random((64, 16, 16, 3))
y =  np.random.random((64, 16, 16, 3))

mygen = JoinedGen(x1, x2, x3, y, 16)

model = CNNmodel(16, 16, 3)
model = model.build_model()

loss = tf.keras.losses.Huber()
metric = ['accuracy']
optimizer = tf.keras.optimizers.Adam(learning_rate=0.0001)

model.compile(optimizer=optimizer,
              loss=loss,
              metrics=metric)


history = model.fit(mygen,
                    epochs=2,
                    batch_size=16)

mygen_preds = JoinedGen(x1, x2, x3,y, 16, preds=True)

现在，如果我打电话：

preds = model.predict(mygen_preds)

我收到 preds 形状：

(8, 16, 16, 3)

如果我打电话：

preds = model.predict([mygen_preds.gen1, mygen_preds.gen2, mygen_preds.gen3],
                      batch_size=1)

我收到形状：

(128, 16, 16, 3)

但我期望的形状是：

(64, 16, 16, 3)

，因为

mygen_preds.gen1.shape

（以及所有其余数组）是

(64, 16, 16, 3)

Answer 1

首先，您目前没有在

get_item

函数中指定批量大小。

您当前仅添加一个新维度作为批次维度，但您可能应该找到一种指定批次维度的好方法。

这与第一个维度始终不同而批量大小恒定有关。

def __getitem__(self, i):
    # Find batch start and end
    start = self.batch_size * i
    end = self.batch_size * (i + 1)

    # Ensure you dont out of range index the indices
    if end >= len(self.indices): end = len(self.indices) - 1

    x1 = self.gen1[start:end, :, :, :]
    x2 = self.gen2[start:end, :, :, :]
    x3 = self.gen3[start:end, :, :, :]
    y = self.target_gen[start:end, :, :, :]
    
    if self.preds:
        return [x1, x2, x3]
    return [x1, x2, x3], y

预期输出应为 [16,16,16,3]，因为第一个维度应等于批次维度。