Numba：双重释放或损坏（！prev）已中止（核心已转储）

Question

我正在尝试使用numba加快以下功能。

import numpy as np
from numba import jit, prange

@jit(nopython=True, parallel=True)
def find_reg_numba(states):
    reg = []    
    states_sum = np.sum(states, axis=1)
    for i in prange(states.shape[0]):
        if states_sum[i] > 0 and states_sum[i] < 5:
            reg.append(states[i])
    return reg

states使用以下功能生成

def generate_states(size):
    # size is a natural number
    states = np.array(list(map(list, itertools.product([0., 1.], repeat = size))))
    return states

[当我尝试使用find_reg功能时，出现以下错误跟踪。

double free or corruption (!prev)
Aborted (core dumped)

我的数字版本是0.48.0。

如何解决此问题？

Answer 1

不确定您的代码为什么会产生错误。相关错误信息为：

• numpy double free or corruption (!prev) segmentation fault

• Error in `python': double free or corruption (!prev)

• Python double free error for huge datasets

但是，这些被证明没有帮助。

这是find_reg_numba的另一种Numba版本，其中：

1. 无错误运行
2. 与不带Numba的原始代码产生相同的结果（即，仅对Numba产生原始错误）。>>

代码重构

import numpy as np
from numba import jit
import itertools

@jit(nopython=True, parallel=True)
def find_reg_numba(states):
    states_sum = np.sum(states, axis=1)

    # Find indexes satisfying condition using np.where as described https://www.geeksforgeeks.org/numpy-where-in-python/
    indexes = np.where((states_sum > 0) & (states_sum < 5))
    return states[indexes]

def generate_states(size):
    # size is a natural number
    states = np.array(list(map(list, itertools.product([0., 1.], repeat = size))))
    return states
测试

for size in range(10):
  s = generate_states(size)
  r  = find_reg_numba(s)
  print(f'Size: {size}\n Result: \n{r}')
结果

 Size: 0
 Result:
[]
Size: 1
 Result:
[[1.]]
Size: 2
 Result:
[[0. 1.]
 [1. 0.]
 [1. 1.]]
Size: 3
 Result:
[[0. 0. 1.]
 [0. 1. 0.]
 [0. 1. 1.]
 [1. 0. 0.]
 [1. 0. 1.]
 [1. 1. 0.]
 [1. 1. 1.]]
Size: 4
 Result:
[[0. 0. 0. 1.]
 [0. 0. 1. 0.]
 [0. 0. 1. 1.]
 [0. 1. 0. 0.]
 [0. 1. 0. 1.]
 [0. 1. 1. 0.]
 [0. 1. 1. 1.]
 [1. 0. 0. 0.]
 [1. 0. 0. 1.]
 [1. 0. 1. 0.]
 [1. 0. 1. 1.]
 [1. 1. 0. 0.]
 [1. 1. 0. 1.]
 [1. 1. 1. 0.]
 [1. 1. 1. 1.]]
Size: 5
 Result:
[[0. 0. 0. 0. 1.]
 [0. 0. 0. 1. 0.]
 [0. 0. 0. 1. 1.]
 [0. 0. 1. 0. 0.]
 [0. 0. 1. 0. 1.]
 [0. 0. 1. 1. 0.]
 [0. 0. 1. 1. 1.]
 [0. 1. 0. 0. 0.]
 [0. 1. 0. 0. 1.]
 [0. 1. 0. 1. 0.]
 [0. 1. 0. 1. 1.]
 [0. 1. 1. 0. 0.]
 [0. 1. 1. 0. 1.]
 [0. 1. 1. 1. 0.]
 [0. 1. 1. 1. 1.]
 [1. 0. 0. 0. 0.]
 [1. 0. 0. 0. 1.]
 [1. 0. 0. 1. 0.]
 [1. 0. 0. 1. 1.]
 [1. 0. 1. 0. 0.]
 [1. 0. 1. 0. 1.]
 [1. 0. 1. 1. 0.]
 [1. 0. 1. 1. 1.]
 [1. 1. 0. 0. 0.]
 [1. 1. 0. 0. 1.]
 [1. 1. 0. 1. 0.]
 [1. 1. 0. 1. 1.]
 [1. 1. 1. 0. 0.]
 [1. 1. 1. 0. 1.]
 [1. 1. 1. 1. 0.]]
Size: 6
 Result:
[[0. 0. 0. 0. 0. 1.]
 [0. 0. 0. 0. 1. 0.]
 [0. 0. 0. 0. 1. 1.]
 [0. 0. 0. 1. 0. 0.]
 [0. 0. 0. 1. 0. 1.]
 [0. 0. 0. 1. 1. 0.]
 [0. 0. 0. 1. 1. 1.]
 [0. 0. 1. 0. 0. 0.]
 [0. 0. 1. 0. 0. 1.]
 [0. 0. 1. 0. 1. 0.]
 [0. 0. 1. 0. 1. 1.]
 [0. 0. 1. 1. 0. 0.]
 [0. 0. 1. 1. 0. 1.]
 [0. 0. 1. 1. 1. 0.]
 [0. 0. 1. 1. 1. 1.]
 [0. 1. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 1.]
 [0. 1. 0. 0. 1. 0.]
 [0. 1. 0. 0. 1. 1.]
 [0. 1. 0. 1. 0. 0.]
 [0. 1. 0. 1. 0. 1.]
 [0. 1. 0. 1. 1. 0.]
 [0. 1. 0. 1. 1. 1.]
 [0. 1. 1. 0. 0. 0.]
 [0. 1. 1. 0. 0. 1.]
 [0. 1. 1. 0. 1. 0.]
 [0. 1. 1. 0. 1. 1.]
 [0. 1. 1. 1. 0. 0.]
 [0. 1. 1. 1. 0. 1.]
 [0. 1. 1. 1. 1. 0.]
 [1. 0. 0. 0. 0. 0.]
 [1. 0. 0. 0. 0. 1.]
 [1. 0. 0. 0. 1. 0.]
 [1. 0. 0. 0. 1. 1.]
 [1. 0. 0. 1. 0. 0.]
 [1. 0. 0. 1. 0. 1.]
 [1. 0. 0. 1. 1. 0.]
 [1. 0. 0. 1. 1. 1.]
 [1. 0. 1. 0. 0. 0.]
 [1. 0. 1. 0. 0. 1.]
 [1. 0. 1. 0. 1. 0.]
 [1. 0. 1. 0. 1. 1.]
 [1. 0. 1. 1. 0. 0.]
 [1. 0. 1. 1. 0. 1.]
 [1. 0. 1. 1. 1. 0.]
 [1. 1. 0. 0. 0. 0.]
 [1. 1. 0. 0. 0. 1.]
 [1. 1. 0. 0. 1. 0.]
 [1. 1. 0. 0. 1. 1.]
 [1. 1. 0. 1. 0. 0.]
 [1. 1. 0. 1. 0. 1.]
 [1. 1. 0. 1. 1. 0.]
 [1. 1. 1. 0. 0. 0.]
 [1. 1. 1. 0. 0. 1.]
 [1. 1. 1. 0. 1. 0.]
 [1. 1. 1. 1. 0. 0.]]
Size: 7
 Result:
[[0. 0. 0. 0. 0. 0. 1.]
 [0. 0. 0. 0. 0. 1. 0.]
 [0. 0. 0. 0. 0. 1. 1.]
 [0. 0. 0. 0. 1. 0. 0.]
 [0. 0. 0. 0. 1. 0. 1.]
 [0. 0. 0. 0. 1. 1. 0.]
 [0. 0. 0. 0. 1. 1. 1.]
 [0. 0. 0. 1. 0. 0. 0.]
 [0. 0. 0. 1. 0. 0. 1.]
 [0. 0. 0. 1. 0. 1. 0.]
 [0. 0. 0. 1. 0. 1. 1.]
 [0. 0. 0. 1. 1. 0. 0.]
 [0. 0. 0. 1. 1. 0. 1.]
 [0. 0. 0. 1. 1. 1. 0.]
 [0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 1. 0. 0. 0. 0.]
 [0. 0. 1. 0. 0. 0. 1.]
 [0. 0. 1. 0. 0. 1. 0.]
 [0. 0. 1. 0. 0. 1. 1.]
 [0. 0. 1. 0. 1. 0. 0.]
 [0. 0. 1. 0. 1. 0. 1.]
 [0. 0. 1. 0. 1. 1. 0.]
 [0. 0. 1. 0. 1. 1. 1.]
 [0. 0. 1. 1. 0. 0. 0.]
 [0. 0. 1. 1. 0. 0. 1.]
 [0. 0. 1. 1. 0. 1. 0.]
 [0. 0. 1. 1. 0. 1. 1.]
 [0. 0. 1. 1. 1. 0. 0.]
 [0. 0. 1. 1. 1. 0. 1.]
 [0. 0. 1. 1. 1. 1. 0.]
 [0. 1. 0. 0. 0. 0. 0.]
 [0. 1. 0. 0. 0. 0. 1.]
 [0. 1. 0. 0. 0. 1. 0.]
 [0. 1. 0. 0. 0. 1. 1.]
 [0. 1. 0. 0. 1. 0. 0.]
 [0. 1. 0. 0. 1. 0. 1.]
 [0. 1. 0. 0. 1. 1. 0.]
 [0. 1. 0. 0. 1. 1. 1.]
 [0. 1. 0. 1. 0. 0. 0.]
 [0. 1. 0. 1. 0. 0. 1.]
 [0. 1. 0. 1. 0. 1. 0.]
 [0. 1. 0. 1. 0. 1. 1.]
 [0. 1. 0. 1. 1. 0. 0.]
 [0. 1. 0. 1. 1. 0. 1.]
 [0. 1. 0. 1. 1. 1. 0.]
 [0. 1. 1. 0. 0. 0. 0.]
 [0. 1. 1. 0. 0. 0. 1.]
 [0. 1. 1. 0. 0. 1. 0.]
 [0. 1. 1. 0. 0. 1. 1.]
 [0. 1. 1. 0. 1. 0. 0.]
 [0. 1. 1. 0. 1. 0. 1.]
 [0. 1. 1. 0. 1. 1. 0.]
 [0. 1. 1. 1. 0. 0. 0.]
 [0. 1. 1. 1. 0. 0. 1.]
 [0. 1. 1. 1. 0. 1. 0.]
 [0. 1. 1. 1. 1. 0. 0.]
 [1. 0. 0. 0. 0. 0. 0.]
 [1. 0. 0. 0. 0. 0. 1.]
 [1. 0. 0. 0. 0. 1. 0.]
 [1. 0. 0. 0. 0. 1. 1.]
 [1. 0. 0. 0. 1. 0. 0.]
 [1. 0. 0. 0. 1. 0. 1.]
 [1. 0. 0. 0. 1. 1. 0.]
 [1. 0. 0. 0. 1. 1. 1.]
 [1. 0. 0. 1. 0. 0. 0.]
 [1. 0. 0. 1. 0. 0. 1.]
 [1. 0. 0. 1. 0. 1. 0.]
 [1. 0. 0. 1. 0. 1. 1.]
 [1. 0. 0. 1. 1. 0. 0.]
 [1. 0. 0. 1. 1. 0. 1.]
 [1. 0. 0. 1. 1. 1. 0.]
 [1. 0. 1. 0. 0. 0. 0.]
 [1. 0. 1. 0. 0. 0. 1.]
 [1. 0. 1. 0. 0. 1. 0.]
 [1. 0. 1. 0. 0. 1. 1.]
 [1. 0. 1. 0. 1. 0. 0.]
 [1. 0. 1. 0. 1. 0. 1.]
 [1. 0. 1. 0. 1. 1. 0.]
 [1. 0. 1. 1. 0. 0. 0.]
 [1. 0. 1. 1. 0. 0. 1.]
 [1. 0. 1. 1. 0. 1. 0.]
 [1. 0. 1. 1. 1. 0. 0.]
 [1. 1. 0. 0. 0. 0. 0.]
 [1. 1. 0. 0. 0. 0. 1.]
 [1. 1. 0. 0. 0. 1. 0.]
 [1. 1. 0. 0. 0. 1. 1.]
 [1. 1. 0. 0. 1. 0. 0.]
 [1. 1. 0. 0. 1. 0. 1.]
 [1. 1. 0. 0. 1. 1. 0.]
 [1. 1. 0. 1. 0. 0. 0.]
 [1. 1. 0. 1. 0. 0. 1.]
 [1. 1. 0. 1. 0. 1. 0.]
 [1. 1. 0. 1. 1. 0. 0.]
 [1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 0. 0. 0. 1.]
 [1. 1. 1. 0. 0. 1. 0.]
 [1. 1. 1. 0. 1. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0.]]
Size: 8
 Result:
[[0. 0. 0. ... 0. 0. 1.]
 [0. 0. 0. ... 0. 1. 0.]
 [0. 0. 0. ... 0. 1. 1.]
 ...
 [1. 1. 1. ... 1. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]]
Size: 9
 Result:
[[0. 0. 0. ... 0. 0. 1.]
 [0. 0. 0. ... 0. 1. 0.]
 [0. 0. 0. ... 0. 1. 1.]
 ...
 [1. 1. 1. ... 0. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]
 [1. 1. 1. ... 0. 0. 0.]]