Milvus 数据库无法加载集合

问题描述 投票:0回答:1

我在 docker 中运行 milvusdb,该数据库中有 18K+ 条记录。虚拟机的存储已满,我停止了 docker 并执行

system prune
删除未使用的资源。我删除了临时日志文件。

Docker 设置如下。

Skn@Skn:~/milvusDB$ sudo docker-compose ps
      Name                     Command                  State                          Ports
----------------------------------------------------------------------------------------------------------------
attu                docker-entrypoint.sh /bin/ ...   Up             0.0.0.0:8000->3000/tcp,:::8000->3000/tcp
milvus-etcd         etcd -advertise-client-url ...   Up             2379/tcp, 2380/tcp
milvus-minio        /usr/bin/docker-entrypoint ...   Up (healthy)   9000/tcp
milvus-standalone   /tini -- milvus run standalone   Up             0.0.0.0:19530->19530/tcp,:::19530->19530/tcp
Skn@Skn:~/milvusDB$ sudo docker ps -a
CONTAINER ID   IMAGE                                      COMMAND                  CREATED          STATUS                    PORTS                                           NAMES
43aec8f43837   zilliz/attu:v2.2.6                         "docker-entrypoint.s…"   33 minutes ago   Up 33 minutes             0.0.0.0:8000->3000/tcp, :::8000->3000/tcp       attu
8ba67d8bdc52   milvusdb/milvus:v2.4.4                     "/tini -- milvus run…"   33 minutes ago   Up 33 minutes             0.0.0.0:19530->19530/tcp, :::19530->19530/tcp   milvus-standalone
3957f3277d7c   minio/minio:RELEASE.2020-12-03T00-03-10Z   "/usr/bin/docker-ent…"   33 minutes ago   Up 33 minutes (healthy)   9000/tcp                                        milvus-minio
78ff48fa2acb   quay.io/coreos/etcd:v3.5.5                 "etcd -advertise-cli…"   33 minutes ago   Up 33 minutes             2379-2380/tcp                                   milvus-etcd

之后,我在容器中运行 milvus 并尝试加载数据。

milvus_collection = "all_products_collection"
from pymilvus import MilvusClient, Collection, connections
connections.connect(host="127.0.0.1", port=19530)
collection = Collection(milvus_collection)
collection.load()
RPC error: [get_loading_progress], <MilvusException: (code=101, message=collection not loaded[collection=451005601884144622])>, <Time:{'RPC start': '2024-11-05 17:31:26.212114', 'RPC error': '2024-11-05 17:31:26.214213'}>
RPC error: [wait_for_loading_collection], <MilvusException: (code=101, message=collection not loaded[collection=451005601884144622])>, <Time:{'RPC start': '2024-11-05 17:21:25.789017', 'RPC error': '2024-11-05 17:31:26.214354'}>
RPC error: [load_collection], <MilvusException: (code=101, message=collection not loaded[collection=451005601884144622])>, <Time:{'RPC start': '2024-11-05 17:21:25.772310', 'RPC error': '2024-11-05 17:31:26.214414'}>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/orm/collection.py", line 424, in load
    conn.load_collection(
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 147, in handler
    raise e from e
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 143, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 182, in handler
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 122, in handler
    raise e from e
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 87, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 1148, in load_collection
    self.wait_for_loading_collection(collection_name, timeout, is_refresh=_refresh)
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 147, in handler
    raise e from e
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 143, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 182, in handler
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 122, in handler
    raise e from e
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 87, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 1168, in wait_for_loading_collection
    progress = self.get_loading_progress(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 147, in handler
    raise e from e
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 143, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 182, in handler
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 122, in handler
    raise e from e
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 87, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 1267, in get_loading_progress
    check_status(response.status)
  File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/client/utils.py", line 62, in check_status
    raise MilvusException(status.code, status.reason, status.error_code)
pymilvus.exceptions.MilvusException: <MilvusException: (code=101, message=collection not loaded[collection=451005601884144622])>

当我检查 milvusdb 独立容器日志时,它给出了显示集合的错误。

{"log":"[2024/11/06 03:47:28.041 +00:00] [INFO] [querycoordv2/services.go:56] [\"show collections request received\"] [traceID=70b08d84e33659c01ee8ad00aba85bdc] [collections=\"[451005601884144622]\"]\n","stream":"stdout","time":"2024-11-06T03:47:28.041529283Z"}
{"log":"[2024/11/06 03:47:28.041 +00:00] [WARN] [querycoordv2/services.go:106] [\"show collection failed\"] [collectionID=451005601884144622] [error=\"collection not loaded[collection=451005601884144622]\"]\n","stream":"stdout","time":"2024-11-06T03:47:28.041593484Z"}
{"log":"[2024/11/06 03:47:28.041 +00:00] [WARN] [proxy/util.go:1298] [\"fail to show collections\"] [collectionID=451005601884144622] [error=\"collection not loaded[collection=451005601884144622]\"]\n","stream":"stdout","time":"2024-11-06T03:47:28.041665585Z"}

在负载收集期间,可能会请求

show collections
并且失败。

我现在如何加载该集合?

python vector load vector-database milvus
1个回答
0
投票

在发布某个集合后,我在 Docker 上运行 milvus-standalone 时遇到了同样的问题。通过增加 milvus.yaml 配置文件中的

loadTimeoutSeconds
解决了。

© www.soinside.com 2019 - 2024. All rights reserved.