我有一个 numpy 数组，主要填充实数，但其中也有一些
nan如何将
nan无需循环：
print(a)
[[ 0.93230948 nan 0.47773439 0.76998063]
[ 0.94460779 0.87882456 0.79615838 0.56282885]
[ 0.94272934 0.48615268 0.06196785 nan]
[ 0.64940216 0.74414127 nan nan]]
#Obtain mean of columns as you need, nanmean is convenient.
col_mean = np.nanmean(a, axis=0)
print(col_mean)
[ 0.86726219 0.7030395 0.44528687 0.66640474]
#Find indices that you need to replace
inds = np.where(np.isnan(a))
#Place column means in the indices. Align the arrays using take
a[inds] = np.take(col_mean, inds[1])
print(a)
[[ 0.93230948 0.7030395 0.47773439 0.76998063]
[ 0.94460779 0.87882456 0.79615838 0.56282885]
[ 0.94272934 0.48615268 0.06196785 0.66640474]
[ 0.64940216 0.74414127 0.44528687 0.66640474]]
仅使用 numpy 执行此操作的标准方法是使用 masked array 模块。
Scipy 是一个相当重的包，它依赖于外部库，因此值得拥有一个仅 numpy 的方法。这借用了@DonaldHobson的答案。
np.nanmean假设你有一个数组
a>>> a
array([[ 0., nan, 10., nan],
[ 1., 6., nan, nan],
[ 2., 7., 12., nan],
[ 3., 8., nan, nan],
[ nan, 9., 14., nan]])
>>> import numpy.ma as ma
>>> np.where(np.isnan(a), ma.array(a, mask=np.isnan(a)).mean(axis=0), a)
array([[ 0. , 7.5, 10. , 0. ],
[ 1. , 6. , 12. , 0. ],
[ 2. , 7. , 12. , 0. ],
[ 3. , 8. , 12. , 0. ],
[ 1.5, 9. , 14. , 0. ]])
请注意，屏蔽数组的均值不需要与
a还要注意如何很好地处理 all-nan 列。平均值为零，因为您取的是零个元素的平均值。使用
nanmean>>> col_mean = np.nanmean(a, axis=0)
/home/praveen/.virtualenvs/numpy3-mkl/lib/python3.4/site-packages/numpy/lib/nanfunctions.py:675: RuntimeWarning: Mean of empty slice
warnings.warn("Mean of empty slice", RuntimeWarning)
>>> inds = np.where(np.isnan(a))
>>> a[inds] = np.take(col_mean, inds[1])
>>> a
array([[ 0. , 7.5, 10. , nan],
[ 1. , 6. , 12. , nan],
[ 2. , 7. , 12. , nan],
[ 3. , 8. , 12. , nan],
[ 1.5, 9. , 14. , nan]])
解释
将
a>>> ma.array(a, mask=np.isnan(a))
masked_array(data =
[[0.0 -- 10.0 --]
[1.0 6.0 -- --]
[2.0 7.0 12.0 --]
[3.0 8.0 -- --]
[-- 9.0 14.0 --]],
mask =
[[False True False True]
[False False True True]
[False False False True]
[False False True True]
[ True False False True]],
fill_value = 1e+20)
对列取平均值可以得到正确的答案，仅对非屏蔽值进行标准化：
>>> ma.array(a, mask=np.isnan(a)).mean(axis=0)
masked_array(data = [1.5 7.5 12.0 --],
mask = [False False False True],
fill_value = 1e+20)
此外，请注意掩码如何很好地处理all-nan！
np.where逐行平均值
要将
nan>>> a
array([[ 0., 1., 2., 3., nan],
[ nan, 6., 7., 8., 9.],
[ 10., nan, 12., nan, 14.],
[ nan, nan, nan, nan, nan]])
>>> np.where(np.isnan(a), ma.array(a, mask=np.isnan(a)).mean(axis=1), a)
ValueError: operands could not be broadcast together with shapes (4,5) (4,) (4,5)
>>> np.where(np.isnan(a), ma.array(a, mask=np.isnan(a)).mean(axis=1)[:, np.newaxis], a)
array([[ 0. , 1. , 2. , 3. , 1.5],
[ 7.5, 6. , 7. , 8. , 9. ],
[ 10. , 12. , 12. , 12. , 14. ],
[ 0. , 0. , 0. , 0. , 0. ]])
如果partial是您的原始数据，并且replace是包含平均值的相同形状的数组，那么此代码将使用partial中的值（如果存在）。
Complete= np.where(np.isnan(partial),replace,partial)
替代方案：用列插值替换 NaN。
def interpolate_nans(X):
"""Overwrite NaNs with column value interpolations."""
for j in range(X.shape[1]):
mask_j = np.isnan(X[:,j])
X[mask_j,j] = np.interp(np.flatnonzero(mask_j), np.flatnonzero(~mask_j), X[~mask_j,j])
return X
使用示例：
X_incomplete = np.array([[10, 20, 30 ],
[np.nan, 30, np.nan],
[np.nan, np.nan, 50 ],
[40, 50, np.nan ]])
X_complete = interpolate_nans(X_incomplete)
print X_complete
[[10, 20, 30 ],
[20, 30, 40 ],
[30, 40, 50 ],
[40, 50, 50 ]]
我特别将这段代码用于时间序列数据，其中列是属性，行是按时间排序的样本。
这不是很干净，但除了迭代之外我想不出其他方法来做到这一点
#example
a = np.arange(16, dtype = float).reshape(4,4)
a[2,2] = np.nan
a[3,3] = np.nan
indices = np.where(np.isnan(a)) #returns an array of rows and column indices
for row, col in zip(*indices):
a[row,col] = np.mean(a[~np.isnan(a[:,col]), col])
为了扩展唐纳德的答案，我提供了一个最小的例子。假设
aIn [231]: a
Out[231]:
array([[0, 3, 6],
[2, 0, 0]])
In [232]: col_mean = np.nanmean(a, axis=0)
Out[232]: array([ 1. , 1.5, 3. ])
In [228]: np.where(np.equal(a, 0), col_mean, a)
Out[228]:
array([[ 1. , 3. , 6. ],
[ 2. , 1.5, 3. ]])
使用带循环的简单函数：
a=[[0.93230948, np.nan, 0.47773439, 0.76998063],
[0.94460779, 0.87882456, 0.79615838, 0.56282885],
[0.94272934, 0.48615268, 0.06196785, np.nan],
[0.64940216, 0.74414127, np.nan, np.nan],
[0.64940216, 0.74414127, np.nan, np.nan]]
print("------- original array -----")
for aa in a:
print(aa)
# GET COLUMN MEANS:
ta = np.array(a).T.tolist() # transpose the array;
col_means = list(map(lambda x: np.nanmean(x), ta)) # get means;
print("column means:", col_means)
# REPLACE NAN ENTRIES WITH COLUMN MEANS:
nrows = len(a); ncols = len(a[0]) # get number of rows & columns;
for r in range(nrows):
for c in range(ncols):
if np.isnan(a[r][c]):
a[r][c] = col_means[c]
print("------- means added -----")
for aa in a:
print(aa)
输出：
------- original array -----
[0.93230948, nan, 0.47773439, 0.76998063]
[0.94460779, 0.87882456, 0.79615838, 0.56282885]
[0.94272934, 0.48615268, 0.06196785, nan]
[0.64940216, 0.74414127, nan, nan]
[0.64940216, 0.74414127, nan, nan]
column means: [0.82369018599999999, 0.71331494500000003, 0.44528687333333333, 0.66640474000000005]
------- means added -----
[0.93230948, 0.71331494500000003, 0.47773439, 0.76998063]
[0.94460779, 0.87882456, 0.79615838, 0.56282885]
[0.94272934, 0.48615268, 0.06196785, 0.66640474000000005]
[0.64940216, 0.74414127, 0.44528687333333333, 0.66640474000000005]
[0.64940216, 0.74414127, 0.44528687333333333, 0.66640474000000005]
for 循环也可以用列表理解来编写：
new_a = [[col_means[c] if np.isnan(a[r][c]) else a[r][c]
for c in range(ncols) ]
for r in range(nrows) ]
numpy.nan_to_numnanimport numpy as np
a = [
[0.93230948, np.nan, 0.47773439, 0.76998063],
[0.94460779, 0.87882456, 0.79615838, 0.56282885],
[0.94272934, 0.48615268, 0.06196785, np.nan],
[0.64940216, 0.74414127, np.nan, np.nan],
]
a = np.nan_to_num(a, nan=np.nanmean(a, axis=0))
print(a)
# Prints:
# [[0.93230948 0.7030395 0.47773439 0.76998063]
# [0.94460779 0.87882456 0.79615838 0.56282885]
# [0.94272934 0.48615268 0.06196785 0.66640474]
# [0.64940216 0.74414127 0.44528687 0.66640474]]
您可能想尝试这个内置功能：
x = np.array([np.inf, -np.inf, np.nan, -128, 128])
np.nan_to_num(x)
array([ 1.79769313e+308, -1.79769313e+308, 0.00000000e+000,
-1.28000000e+002, 1.28000000e+002])