我在 docker 中运行 milvusdb,该数据库中有 18K+ 条记录。虚拟机的存储已满,我停止了 docker 并执行
system prune
删除未使用的资源。我删除了临时日志文件。
Docker 设置如下。
Skn@Skn:~/milvusDB$ sudo docker-compose ps
Name Command State Ports
----------------------------------------------------------------------------------------------------------------
attu docker-entrypoint.sh /bin/ ... Up 0.0.0.0:8000->3000/tcp,:::8000->3000/tcp
milvus-etcd etcd -advertise-client-url ... Up 2379/tcp, 2380/tcp
milvus-minio /usr/bin/docker-entrypoint ... Up (healthy) 9000/tcp
milvus-standalone /tini -- milvus run standalone Up 0.0.0.0:19530->19530/tcp,:::19530->19530/tcp
Skn@Skn:~/milvusDB$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
43aec8f43837 zilliz/attu:v2.2.6 "docker-entrypoint.s…" 33 minutes ago Up 33 minutes 0.0.0.0:8000->3000/tcp, :::8000->3000/tcp attu
8ba67d8bdc52 milvusdb/milvus:v2.4.4 "/tini -- milvus run…" 33 minutes ago Up 33 minutes 0.0.0.0:19530->19530/tcp, :::19530->19530/tcp milvus-standalone
3957f3277d7c minio/minio:RELEASE.2020-12-03T00-03-10Z "/usr/bin/docker-ent…" 33 minutes ago Up 33 minutes (healthy) 9000/tcp milvus-minio
78ff48fa2acb quay.io/coreos/etcd:v3.5.5 "etcd -advertise-cli…" 33 minutes ago Up 33 minutes 2379-2380/tcp milvus-etcd
之后,我在容器中运行 milvus 并尝试加载数据。
milvus_collection = "all_products_collection"
from pymilvus import MilvusClient, Collection, connections
connections.connect(host="127.0.0.1", port=19530)
collection = Collection(milvus_collection)
collection.load()
RPC error: [get_loading_progress], <MilvusException: (code=101, message=collection not loaded[collection=451005601884144622])>, <Time:{'RPC start': '2024-11-05 17:31:26.212114', 'RPC error': '2024-11-05 17:31:26.214213'}>
RPC error: [wait_for_loading_collection], <MilvusException: (code=101, message=collection not loaded[collection=451005601884144622])>, <Time:{'RPC start': '2024-11-05 17:21:25.789017', 'RPC error': '2024-11-05 17:31:26.214354'}>
RPC error: [load_collection], <MilvusException: (code=101, message=collection not loaded[collection=451005601884144622])>, <Time:{'RPC start': '2024-11-05 17:21:25.772310', 'RPC error': '2024-11-05 17:31:26.214414'}>
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/orm/collection.py", line 424, in load
conn.load_collection(
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 147, in handler
raise e from e
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 143, in handler
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 182, in handler
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 122, in handler
raise e from e
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 87, in handler
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 1148, in load_collection
self.wait_for_loading_collection(collection_name, timeout, is_refresh=_refresh)
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 147, in handler
raise e from e
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 143, in handler
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 182, in handler
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 122, in handler
raise e from e
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 87, in handler
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 1168, in wait_for_loading_collection
progress = self.get_loading_progress(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 147, in handler
raise e from e
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 143, in handler
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 182, in handler
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 122, in handler
raise e from e
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/decorators.py", line 87, in handler
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 1267, in get_loading_progress
check_status(response.status)
File "/home/AsmaSkinMatch/.local/lib/python3.11/site-packages/pymilvus/client/utils.py", line 62, in check_status
raise MilvusException(status.code, status.reason, status.error_code)
pymilvus.exceptions.MilvusException: <MilvusException: (code=101, message=collection not loaded[collection=451005601884144622])>
当我检查 milvusdb 独立容器日志时,它给出了显示集合的错误。
{"log":"[2024/11/06 03:47:28.041 +00:00] [INFO] [querycoordv2/services.go:56] [\"show collections request received\"] [traceID=70b08d84e33659c01ee8ad00aba85bdc] [collections=\"[451005601884144622]\"]\n","stream":"stdout","time":"2024-11-06T03:47:28.041529283Z"}
{"log":"[2024/11/06 03:47:28.041 +00:00] [WARN] [querycoordv2/services.go:106] [\"show collection failed\"] [collectionID=451005601884144622] [error=\"collection not loaded[collection=451005601884144622]\"]\n","stream":"stdout","time":"2024-11-06T03:47:28.041593484Z"}
{"log":"[2024/11/06 03:47:28.041 +00:00] [WARN] [proxy/util.go:1298] [\"fail to show collections\"] [collectionID=451005601884144622] [error=\"collection not loaded[collection=451005601884144622]\"]\n","stream":"stdout","time":"2024-11-06T03:47:28.041665585Z"}
在负载收集期间,可能会请求
show collections
并且失败。
我现在如何加载该集合?
在发布某个集合后,我在 Docker 上运行 milvus-standalone 时遇到了同样的问题。通过增加 milvus.yaml 配置文件中的
loadTimeoutSeconds
解决了。