使用 hwloc 分配 NUMA 内存

Question

我正在尝试使用 hwloc 进行 NUMA 感知内存分配并得到一些奇怪的行为。

我的目标是在不同的 NUMA 节点上分配内存块，因为我需要为一个项目分配内存块。为了验证是否分配了正确数量的内存，我一直在使用 valgrind 的 memcheck 工具。无论我要分配多少元素，该工具总是报告分配的字节数相同，这对我来说没有意义。总体而言，分配数量似乎很高。我知道 hwloc 需要在内部分配一些东西才能正常工作，但这对我来说仍然没有意义。

即使我尝试分配更大的内存块，valgrind 报告的分配字节保持不变。

这是我一直用来在 NUMA 节点 0 上分配内存的代码。

#include <iostream>
#include <hwloc.h>

int main() {
    hwloc_topology_t topo;
    hwloc_topology_init(&topo);
    hwloc_topology_load(topo);

    auto node = hwloc_get_obj_by_type(topo, HWLOC_OBJ_NUMANODE, 0);

    auto mem = hwloc_alloc_membind(topo, SIZE * sizeof(float), node->nodeset,
                    HWLOC_MEMBIND_BIND, HWLOC_MEMBIND_STRICT | HWLOC_MEMBIND_BYNODESET);

    if (mem == NULL) {
        std::cout << "Allocation failed" << std::endl;
    }   

    hwloc_free(topo, mem, SIZE);
    hwloc_topology_destroy(topo);
}

这是 valgrind memcheck 报告的相关部分：

真的希望有人能解释这里发生了什么。

我正在使用 hwloc 2.7.1 和 valgrind 3.15.0

Answer 1

Valgrind 3.15 已经很老了——你能试试更新的吗？

查看您需要使用的一切

 valgrind --leak-check=full --show-reachable=yes --default-suppressions=no

我得到（使用 SIZE 1024）

==1714== HEAP SUMMARY:
==1714==     in use at exit: 2,777 bytes in 4 blocks
==1714==   total heap usage: 224 allocs, 220 frees, 31,139 bytes allocated
==1714== 
==1709== 9 bytes in 1 blocks are definitely lost in loss record 1 of 4
==1709==    at 0x484CBC4: malloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==1709==    by 0x48A060D: ??? (in /usr/local/lib/libhwloc.so.15.6.0)
==1709==    by 0x48751E5: hwloc_topology_load (in /usr/local/lib/libhwloc.so.15.6.0)
==1709==    by 0x202980: main (hwloc.cpp:7)
==1709== 
==1709== 64 bytes in 1 blocks are still reachable in loss record 2 of 4
==1709==    at 0x48500D5: calloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==1709==    by 0x4FFD7B2: ??? (in /lib/libthr.so.3)
==1709==    by 0x4FF5FC9: ??? (in /lib/libthr.so.3)
==1709==    by 0x4FF5139: ??? (in /lib/libthr.so.3)
==1709==    by 0x400B0FC: ??? (in /libexec/ld-elf.so.1)
==1709==    by 0x400938A: ??? (in /libexec/ld-elf.so.1)
==1709==    by 0x4006F88: ??? (in /libexec/ld-elf.so.1)
==1709== 
==1709== 1,040 bytes in 1 blocks are still reachable in loss record 3 of 4
==1709==    at 0x484CBC4: malloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==1709==    by 0x4B57254: ??? (in /lib/libc.so.7)
==1709==    by 0x4B573D2: __cxa_atexit (in /lib/libc.so.7)
==1709==    by 0x4E53A68: ??? (in /usr/local/lib/libze_loader.so.1.8.12)
==1709==    by 0x400B0FC: ??? (in /libexec/ld-elf.so.1)
==1709==    by 0x400938A: ??? (in /libexec/ld-elf.so.1)
==1709==    by 0x4006F88: ??? (in /libexec/ld-elf.so.1)
==1709== 
==1709== 1,664 bytes in 1 blocks are still reachable in loss record 4 of 4
==1709==    at 0x48500D5: calloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==1709==    by 0x4FF5FB8: ??? (in /lib/libthr.so.3)
==1709==    by 0x4FF5139: ??? (in /lib/libthr.so.3)
==1709==    by 0x400B0FC: ??? (in /libexec/ld-elf.so.1)
==1709==    by 0x400938A: ??? (in /libexec/ld-elf.so.1)
==1709==    by 0x4006F88: ??? (in /libexec/ld-elf.so.1)
==1709== 
==1709== LEAK SUMMARY:
==1709==    definitely lost: 9 bytes in 1 blocks
==1709==    indirectly lost: 0 bytes in 0 blocks
==1709==      possibly lost: 0 bytes in 0 blocks
==1709==    still reachable: 2,768 bytes in 3 blocks
==1709==         suppressed: 0 bytes in 0 blocks