为卷积网络的每一层激活层添加可视化 - Keras

Question

我有一个卷积网络（CNN），如下所示。我想为每个层激活层添加可视化，如

CNN 的多个层正在执行所需的任务。我只想探测每一层的输出。

def get_model():
    input_shape = (IMG_MODE, img_rows, img_cols)
    model = Sequential()

    model.add(ZeroPadding2D(padding=(1,1), input_shape=input_shape))
    model.add(Conv2D(32, (3, 3), padding = 'valid'))
    model.add(LeakyReLU(alpha=0.01))
    model.add(MaxPooling2D(pool_size=pool_size2))

    ....

    model.add(Dense(nb_classes))
    model.add(Activation('softmax'))

    print(model.summary())
    return model

代码输出：

_________________________________________________________________


   Layer (type)                 Output Shape              Param #   
    =================================================================
    zero_padding2d_1 (ZeroPaddin (None, 1, 114, 94)        0         
    _________________________________________________________________
    conv2d_1 (Conv2D)            (None, 32, 112, 92)       320       
    _________________________________________________________________
    leaky_re_lu_1 (LeakyReLU)    (None, 32, 112, 92)       0         
    _________________________________________________________________
    max_pooling2d_1 (MaxPooling2 (None, 32, 56, 46)        0         
    _________________________________________________________________
    ....        
    _________________________________________________________________
    dense_1 (Dense)              (None, 1024)              8258560   
    _________________________________________________________________
    leaky_re_lu_4 (LeakyReLU)    (None, 1024)              0         
    _________________________________________________________________
    dropout_1 (Dropout)          (None, 1024)              0         
    _________________________________________________________________
    dense_2 (Dense)              (None, 40)                41000     
    _________________________________________________________________
    activation_1 (Activation)    (None, 40)                0         
    =================================================================
    Total params: 8,392,232
    Trainable params: 8,392,232
    Non-trainable params: 0
    _________________________________________________________________
    None
    Train on 320 samples, validate on 80 samples
    Epoch 1/20
     - 18s - loss: 3.7036 - acc: 0.0187 - val_loss: 3.6824 - val_acc: 0.0250
    Epoch 2/20
     - 17s - loss: 3.6903 - acc: 0.0250 - val_loss: 3.6786 - val_acc: 0.0250
    ...
    Epoch 20/20
     - 17s - loss: 0.2067 - acc: 0.9312 - val_loss: 0.9892 - val_acc: 0.7625
    Test score: 0.9891735315322876
    Test accuracy: 0.7625

我尝试使用以下代码来完成我的任务：

 import matplotlib.pyplot as plt
    from keras import models
    layer_outputs = [layer.output for layer in model.layers[:8]]
    activation_model = models.Model(inputs=model.input, outputs=layer_outputs)

    activations = activation_model.predict(img_tensor)

    import matplotlib.pyplot as plt
    plt.matshow(first_layer_activation[0, :, :, 7], cmap='viridis')
    layer_names = []
    for layer in model.layers[:8]:
    layer_names.append(layer.name)
    images_per_row = 16
    for layer_name, layer_activation in zip(layer_names, activations):
    n_features = layer_activation.shape[-1]
    size = layer_activation.shape[1]
    n_cols = n_features // images_per_row
    display_grid = np.zeros((size * n_cols, images_per_row * size))
    for col in range(n_cols):
    for row in range(images_per_row):
    channel_image = layer_activation[0,
    :, :,
    col * images_per_row + row]
    channel_image -= channel_image.mean()
    channel_image /= channel_image.std()
    channel_image *= 64
    channel_image += 128
    channel_image = np.clip(channel_image, 0, 255).astype('uint8')
    display_grid[col * size : (col + 1) * size,
    row * size : (row + 1) * size] = channel_image
    scale = 1. / size
    plt.figure(figsize=(scale * display_grid.shape[1],
    scale * display_grid.shape[0]))
    plt.title(layer_name)
    plt.grid(False)
    plt.imshow(display_grid, aspect='auto', cmap='viridis')

Answer 1

下面是从任何模型的卷积层获取输出并将其可视化的通用方法。这里使用的是Keras的TensorFlow版本； Keras 其他实现的等效代码可能略有不同。

首先需要一个函数来获取模型卷积层的输出：

#Setting index = -1 means we select the last convolutional layer
def get_conv_output(model, index = -1): 
    layers = model.layers
    result = [layer.output for layer in layers 
             if type(layer) is tf.keras.layers.Conv2D][index]
        
    return result

此输出将是形状为

(batch_size, height, width, number_of_channels)

的四维张量。 注意：在对网络的单个输入图像进行评估的情况下，

batch_size

为1。

接下来我们需要一个函数，为单个数据元素（单个输入图像）构建卷积层激活图的网格。它将构建一个近乎正方形的图像，它将成为这些地图的网格：

def maps_to_grid(output):
    #Calculate the number of rows and columns needed to arrange
    #the activation maps into a nearly-square grid. The number of 
    #maps is the number of channels in the convolutional layer output
    num_maps = int(output.shape[-1])
    num_columns = math.ceil(num_maps ** 0.5)
    num_rows = math.ceil(num_maps / num_columns)
    
    #The end of the set of activation maps may need to be padded 
    #with zeroes to fit it into the grid
    num_zeros = num_rows * num_columns - num_maps 
    zeros_shape = [int(i) for i in output.shape]
    zeros_shape[-1] = num_zeros
    zeros = tf.zeros(zeros_shape,
                     dtype = tf.float32,
                     name = None)
    
    #Pad the activation maps with zeroes, concatenating along the
    #channels dimension
    padded_output = tf.concat([output, zeros], -1)
    len, width, depth = [s for s in padded_output.shape]
    
    #Unstack the padded activation maps and construct the grid
    map_stack = tf.unstack(padded_output, axis = 2)
    row_stacks = [tf.concat(map_stack[i : i + num_columns], axis = 1) 
                  for i in range(0, num_columns * num_rows, num_columns)]
    result = tf.concat(row_stacks, axis = 0)
        
    return result 

一旦你有了这些功能，你就可以得到网格如下：

activation_map_grid_tensor = maps_to_grid(get_conv_output(model)[0])

Index

0

是必需的，因为

maps_to_grid

适用于单个图像的激活图，因此我们选择批次的第一个元素。 现在您可以评估张量，并用例如显示它

cv2.imshow()

此方法取自 https://github.com/cyberneuron/RT-CNN-Vis，这是一个 CNN 可视化平台。人们可能还会发现直接从那里获取代码更容易。