无法使用自定义正则化类加载经过训练的 keras 模型

Question

我正在按照 Keras 中的教程使用自己的数据集训练 PointNet3D 对象分类模型：https://keras.io/examples/vision/pointnet/#point-cloud-classification-with-pointnet

现在对于训练部分，我已经能够很好地完成所有事情，但训练后我在加载经过训练的模型时遇到问题。 我认为主要问题是下面这部分，当我保存模型时，

OrthogonalRegularizer

类对象可能无法正确注册：

@keras.saving.register_keras_serializable('OrthogonalRegularizer')
class OrthogonalRegularizer(keras.regularizers.Regularizer):

    def __init__(self, num_features, **kwargs):
        super(OrthogonalRegularizer, self).__init__(**kwargs)
        self.num_features = num_features
        self.l2reg = 0.001
        self.eye = tf.eye(num_features)

    def __call__(self, x):
        x = tf.reshape(x, (-1, self.num_features, self.num_features))
        xxt = tf.tensordot(x, x, axes=(2, 2))
        xxt = tf.reshape(xxt, (-1, self.num_features, self.num_features))
        return tf.math.reduce_sum(self.l2reg * tf.square(xxt - self.eye))

    def get_config(self):
        config = {}
        config.update({"num_features": self.num_features, "l2reg": self.l2reg, "eye": self.eye})
        return config

def tnet(inputs, num_features):
    # Initialise bias as the identity matrix
    bias = keras.initializers.Constant(np.eye(num_features).flatten())
    reg = OrthogonalRegularizer(num_features)

    x = conv_bn(inputs, 32)
    x = conv_bn(x, 64)
    x = conv_bn(x, 512)
    x = layers.GlobalMaxPooling1D()(x)
    x = dense_bn(x, 256)
    x = dense_bn(x, 128)
    x = layers.Dense(
        num_features * num_features,
        kernel_initializer="zeros",
        bias_initializer=bias,
        activity_regularizer=reg,
    )(x)
    feat_T = layers.Reshape((num_features, num_features))(x)
    # Apply affine transformation to input features
    return layers.Dot(axes=(2, 1))([inputs, feat_T])

训练后，当我尝试通过以下方式加载模型时，我看到以下错误：

model.save('my_model.h5')
model = keras.models.load_model('my_model.h5', custom_objects={'OrthogonalRegularizer': OrthogonalRegularizer})

错误信息：

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-18-05f700f433a8> in <cell line: 2>()
      1 model.save('my_model.h5')
----> 2 model = keras.models.load_model('my_model.h5', custom_objects={'OrthogonalRegularizer': OrthogonalRegularizer})

2 frames
/usr/local/lib/python3.10/dist-packages/keras/src/saving/saving_api.py in load_model(filepath, custom_objects, compile, safe_mode, **kwargs)
    260 
    261     # Legacy case.
--> 262     return legacy_sm_saving_lib.load_model(
    263         filepath, custom_objects=custom_objects, compile=compile, **kwargs
    264     )

/usr/local/lib/python3.10/dist-packages/keras/src/utils/traceback_utils.py in error_handler(*args, **kwargs)
     68             # To get the full stack trace, call:
     69             # `tf.debugging.disable_traceback_filtering()`
---> 70             raise e.with_traceback(filtered_tb) from None
     71         finally:
     72             del filtered_tb

/usr/local/lib/python3.10/dist-packages/keras/src/engine/base_layer.py in from_config(cls, config)
    868             return cls(**config)
    869         except Exception as e:
--> 870             raise TypeError(
    871                 f"Error when deserializing class '{cls.__name__}' using "
    872                 f"config={config}.\n\nException encountered: {e}"

TypeError: Error when deserializing class 'Dense' using config={'name': 'dense_2', 
'trainable': True, 'dtype': 'float32', 'units': 9, 'activation': 'linear', 'use_bias': 

True, 'kernel_initializer': {'module': 'keras.initializers', 'class_name': 'Zeros', 
'config': {}, 'registered_name': None}, 'bias_initializer': {'module': 
'keras.initializers', 'class_name': 'Constant', 'config': {'value': {'class_name': 
'__numpy__', 'config': {'value': [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0], 'dtype': 
'float64'}}}, 'registered_name': None}, 'kernel_regularizer': None, 'bias_regularizer': 
None, 'activity_regularizer': {'module': None, 'class_name': 'OrthogonalRegularizer', 
'config': {'num_features': 3, 'l2reg': 0.001, 'eye': {'class_name': '__tensor__', 
'config': {'value': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]], 'dtype': 
'float32'}}}, 'registered_name': 'OrthogonalRegularizer>OrthogonalRegularizer'}, 
'kernel_constraint': None, 'bias_constraint': None}.

Exception encountered: object.__init__() takes exactly one argument (the instance to initialize)

到目前为止我所理解的是，在保存时我无法正确保存

OrthogonalRegularizer

类对象。请让我知道我做错了什么。

代码的最小版本已上传到此协作笔记本中： https://colab.research.google.com/drive/1akpfoOBVAWThsZl7moYywuZIuXt_vWCU?usp=sharing

一个可能的类似问题是这样的：在 Keras 中加载自定义正则化器

Answer 1

您无需致电

super(OrthogonalRegularizer, self).__init__(**kwargs)

因为 keras.regularizers.Regularizer 中的构造函数未定义 也不需要存储不可序列化的张量“eye”：self.eye。此外，最好在靠近使用位置的地方创建这个张量。 修改后的代码应如下所示：

@keras.saving.register_keras_serializable('OrthogonalRegularizer')
class OrthogonalRegularizer(keras.regularizers.Regularizer):

def __init__(self, num_features, **kwargs):
    self.num_features = num_features
    self.l2reg = 0.001

def __call__(self, x):
    x = tf.reshape(x, (-1, self.num_features, self.num_features))
    xxt = tf.tensordot(x, x, axes=(2, 2))
    xxt = tf.reshape(xxt, (-1, self.num_features, self.num_features))
    eye = tf.eye(self.num_features)
    return tf.math.reduce_sum(self.l2reg * tf.square(xxt - eye))

def get_config(self):
    return {"num_features": self.num_features, "l2reg": self.l2reg}

此外，最好使用更新类型的保存和加载模型

model.save('my_model')
model = keras.models.load_model('my_model', custom_objects={'OrthogonalRegularizer': OrthogonalRegularizer})