nxruntime 推理故障排除 - X num_dims 与 W num_dims 不匹配

Question

使用此处找到的 nnhash.py 脚本：https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX

# Copyright 2021 Asuhariet Ygvar
# 
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# 
# http://www.apache.org/licenses/LICENSE-2.0
# 
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
# or implied. See the License for the specific language governing
# permissions and limitations under the License.

import sys
import onnxruntime
import numpy as np
from PIL import Image

# Load ONNX model
session = onnxruntime.InferenceSession(sys.argv[1])

# Load output hash matrix
seed1 = open(sys.argv[2], 'rb').read()[128:]
seed1 = np.frombuffer(seed1, dtype=np.float32)
seed1 = seed1.reshape([96, 128])

# Preprocess image
image = Image.open(sys.argv[3]).convert('RGB')
image = image.resize([360, 360])
arr = np.array(image).astype(np.float32) / 255.0
arr = arr * 2.0 - 1.0
arr = arr.transpose(2, 0, 1).reshape([1, 3, 360, 360])

# Run model
inputs = {session.get_inputs()[0].name: arr}
outs = session.run(None, inputs)

# Convert model output to hex hash
hash_output = seed1.dot(outs[0].flatten())
hash_bits = ''.join(['1' if it >= 0 else '0' for it in hash_output])
hash_hex = '{:0{}x}'.format(int(hash_bits, 2), len(hash_bits) // 4)

print(hash_hex)

当我尝试使用以下命令执行时：

python nnhash.py ../model.onnx ../seed1.dat ../1.png

onnxruntime.capi.onnxruntime_pybind11_state.Fail：[ONNXRuntimeError]： 1：失败：运行 FusedConv 节点时返回非零状态代码。 名称：'' 状态消息：X num_dims 与 W num_dims 不匹配。 X： {1,1280,1,1} 西：{500}

我附上了 Netron 层输出的样子：https://i.imgur.com/EeVItQ2.jpeg（将其作为链接发送，因为实际图像非常长）

据我所知，在最底部，叶子部分之前有一个 W:{500}。我确信这就是导致问题的原因，我只是不确定我需要做什么才能处理输入图像，以便它可以很好地流经模型。

编辑：我发现一个旧的 model.onnx 似乎工作正常，最后几层有点不同，我认为这就是问题所在。我不知道该怎么做才能让脚本与较新的 model.onnx 文件一起使用。

旧：

新：

Answer 1

我发现视觉库还完成了额外的转换，所以我编写了一个 python 脚本以便只修改最后几层：

import onnx
from onnx import helper, shape_inference, numpy_helper, TensorProto
import numpy as np

def is_node_used(node_name, graph):
    for node in graph.node:
        if node_name in node.input:
            return True
    return False

# Load the new ONNX model
model_path = 'model.onnx'
new_model = onnx.load(model_path)

# Load the old ONNX model to use as a reference
old_model_path = 'old-model.onnx'
old_model = onnx.load(old_model_path)

# Extract the graph from the new model
graph = new_model.graph

# Remove the last layer(s) from the new model
for node in graph.node:
    if node.output[0] == 'leaf/logits':
        graph.node.remove(node)

# Create a new last layer that matches the old model's configuration
# Define the weight and bias for the new Conv layer
W_conv_last = numpy_helper.from_array(
    np.random.rand(128, 500, 1, 1).astype(np.float32), name='W_conv_last'
)
B_conv_last = numpy_helper.from_array(
    np.random.rand(128).astype(np.float32), name='B_conv_last'
)

# Add weight and bias initializers to the graph
graph.initializer.append(W_conv_last)
graph.initializer.append(B_conv_last)

# Create the new Conv node for the last layer
new_conv_last = helper.make_node(
    'Conv',
    inputs=['relu_output', 'W_conv_last', 'B_conv_last'],
    outputs=['leaf/logits'],
    kernel_shape=[1, 1],
    pads=[0, 0, 0, 0],
    strides=[1, 1],
    dilations=[1, 1],
)

# Define weights and biases for the modified convolutional layer
W_conv_mod = numpy_helper.from_array(
    np.random.rand(500, 1280, 1, 1).astype(np.float32), name='W_conv_mod'
)
B_conv_mod = numpy_helper.from_array(
    np.random.rand(500).astype(np.float32), name='B_conv_mod'
)

# Add weight and bias initializers to the graph
graph.initializer.append(W_conv_mod)
graph.initializer.append(B_conv_mod)

# Create the new Conv node for the modified layer
new_conv_mod = helper.make_node(
    'Conv',
    inputs=['stem/MobilenetV3/GAP_13/mul_1', 'W_conv_mod', 'B_conv_mod'],
    outputs=['conv_output'],
    kernel_shape=[1, 1],
    pads=[0, 0, 0, 0],
    strides=[1, 1],
    dilations=[1, 1],
)

# Create the ReLU node
new_relu = helper.make_node(
    'Relu',
    inputs=['conv_output'],
    outputs=['relu_output'],
)

# Find and remove the old layer with the input 'stem/MobilenetV3/GAP_13/mul_1'
for node in graph.node:
    if 'stem/MobilenetV3/GAP_13/mul_1' in node.input:
        graph.node.remove(node)
        break

# Insert the new Conv node for the modified layer, followed by ReLU and the last Conv node
graph.node.append(new_conv_mod)
graph.node.append(new_relu)
graph.node.append(new_conv_last)

# Remove any floating ReLU nodes
nodes_to_remove = []
for node in graph.node:
    if node.op_type == 'Relu' and not is_node_used(node.output[0], graph):
        nodes_to_remove.append(node)

for node in nodes_to_remove:
    graph.node.remove(node)

# Save the modified ONNX model
onnx.save(new_model, 'modified_new_model.onnx')

这将采用新模型并使最后几层看起来像旧模型的样子并保存新模型。现在它按预期工作了。