我在尝试使用 OpenAI 嵌入在 Milvus 中搜索相似向量时遇到数据类型不匹配错误。尽管遵循架构设置和索引指南,但仍会发生错误。
我正在使用:
架构设置:
const schema = [
{
name: "primary_key",
description: "primary key",
data_type: DataType.Int64,
is_primary_key: true,
autoID: true,
},
{
name: "vector",
description: "text vector",
data_type: DataType.FloatVector,
dim: dim,
},
{
name: "raw",
description: "raw text for vector",
data_type: DataType.VarChar,
max_length: 1000,
}
];
创建索引:
const res = await milvusClient.createIndex({
collection_name: orgName,
field_name: "vector",
index_name: "vector_index",
index_type: "FLAT",
metric_type: "L2",
});
搜索查询:
const res = await milvusClient.search({
collection_name: orgName,
vector: vectorQuery,
limit: 1,
output_fields: ["raw"],
});
我收到以下错误消息:
{
status: {
error_code: 'UnexpectedError',
reason: 'fail to search on all shard leaders, err=All attempts results:
attempt #1:code: UnexpectedError, error: fail to Search, QueryNode ID=2,
reason=stream operation failed: [UnexpectedError] Assert "is_float_data_type == is_float_metric_type"
at /go/src/github.com/milvus-io/milvus/internal/core/src/query/SearchBruteForce.cpp:31
=> [BruteForceSearch] Data type and metric type mis-match
attempt #2:context canceled',
code: 0,
retriable: false,
detail: ''
},
results: []
}
我已经尝试了 Node SDK README 中的示例并得到了相同的错误。该错误表明数据类型和度量类型之间不匹配,但我使用的是 FloatVector 数据类型,它应该与 L2 距离度量兼容。
如何解决此数据类型和指标类型不匹配的问题?
我认为问题在于矢量数据的结构。 Milvus 对于矢量数据的格式化方式非常讲究。
以下是密集向量和稀疏向量的两个示例:
密集向量:
// Example of properly structured dense vectors
export const vectorsData = [
{
vector: [
0.11878310581111173, 0.9694947902934701, 0.16443679307243175,
0.5484226189097237, 0.9839246709011924, 0.5178387104937776,
0.8716926129208069, 0.5616972243831446,
],
height: 20405,
name: 'zlnmh',
},
// ... more vector entries
];
对于稀疏向量:
import { SparseFloatVector } from "@zilliz/milvus2-sdk-node";
type sparseVectorsDataItem = {
vector: SparseFloatVector;
height: number;
name: string;
};
export const sparseVectorsData: sparseVectorsDataItem[] = [
{
"vector": {"1": 0.1, "100": 0.3, "55": 0.324},
"height": 20405,
"name": "zlnmh"
},
// ... more sparse vector entries
];
当您设置的向量格式与 Milvus 的期望不匹配时,就会发生错误。 特别是使用 OpenAI 嵌入: