我在用ray做并行计算的时候遇到了内存泄露。为了缩小原因范围,我从 python 脚本中删除了除以下代码之外的所有代码:
import ray
print("just a blank script")
仍然存在内存泄漏错误:
==8279== 54 bytes in 1 blocks are definitely lost in loss record 1,697 of 10,519
==8279== at 0x4C30EDB: malloc (vg_replace_malloc.c:309)
==8279== by 0x4F23592: PyObject_Malloc (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F5EAE9: PyUnicode_New (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F675F0: _PyUnicodeWriter_PrepareInternal (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F3806E: PyUnicode_DecodeUTF8Stateful (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F7A9F7: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F7AFFC: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F876AD: _Py_BuildValue_SizeT (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x40BA46D: PyInit_setproctitle (setproctitle.c:174)
==8279== by 0x5053281: _PyImport_LoadDynamicModuleWithSpec (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x5053584: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FAF706: PyCFunction_Call (in /usr/lib64/libpython3.6m.so.1.0)
==8279==
==8279== 56 bytes in 1 blocks are definitely lost in loss record 3,102 of 10,519
==8279== at 0x4C30EDB: malloc (vg_replace_malloc.c:309)
==8279== by 0x4F23592: PyObject_Malloc (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F3BB5C: _PyObject_GC_Malloc (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F4242D: _PyObject_GC_NewVar (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F5645E: PyTuple_New (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FD85E8: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FD8B18: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FD872E: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FD8B18: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FD872E: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FD965C: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FDCAA0: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279==
==8279== 64 bytes in 1 blocks are definitely lost in loss record 3,891 of 10,519
==8279== at 0x4C30EDB: malloc (vg_replace_malloc.c:309)
==8279== by 0x40141AD: dl_open_worker (in /usr/lib64/ld-2.28.so)
==8279== by 0x5E79AB6: _dl_catch_exception (in /usr/lib64/libc-2.28.so)
==8279== by 0x401365D: _dl_open (in /usr/lib64/ld-2.28.so)
==8279== by 0x55B81B9: dlopen_doit (in /usr/lib64/libdl-2.28.so)
==8279== by 0x5E79AB6: _dl_catch_exception (in /usr/lib64/libc-2.28.so)
==8279== by 0x5E79B52: _dl_catch_error (in /usr/lib64/libc-2.28.so)
==8279== by 0x55B8938: _dlerror_run (in /usr/lib64/libdl-2.28.so)
==8279== by 0x55B8259: dlopen@@GLIBC_2.2.5 (in /usr/lib64/libdl-2.28.so)
==8279== by 0x502DE19: _PyImport_FindSharedFuncptr (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x5053239: _PyImport_LoadDynamicModuleWithSpec (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x5053584: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279==
==8279== 144 bytes in 1 blocks are definitely lost in loss record 7,610 of 10,519
==8279== at 0x4C30EDB: malloc (vg_replace_malloc.c:309)
==8279== by 0x4F23592: PyObject_Malloc (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FCF8B2: PyCode_New (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x89FE769: __pyx_pymod_exec__raylet(_object*) (in /root/.local/lib/python3.6/site-packages/ray/_raylet.so)
==8279== by 0x502C2F2: PyModule_ExecDef (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x502C381: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FAF706: PyCFunction_Call (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FF3C41: _PyEval_EvalFrameDefault (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F3FBE7: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4F795DF: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FBDA31: ??? (in /usr/lib64/libpython3.6m.so.1.0)
==8279== by 0x4FEDA69: _PyEval_EvalFrameDefault (in /usr/lib64/libpython3.6m.so.1.0)
我使用以下命令进行内存测试:
PYTHONMALLOC=malloc valgrind --leak-check=full -v --track-origins=yes \
--log-file=$SCRIPT_DIR"/algrind_pytest.log" \
--show-possibly-lost=no \
python test.py
我的环境: python3.6.8 valgrind 3.15.0
有人可以帮忙吗?
我想知道内存泄漏的原因。如果没有问题,如何删除错误报告。