有效地将不均匀的列表列表转换为最小的包含用nan填充的数组

Question

考虑列表清单 l

l = [[1, 2, 3], [1, 2]]

如果我将其转换为a,np.array我将[1, 2, 3]在第一个位置和[1, 2]第二个位置获得一维对象数组.

print(np.array(l))

[[1, 2, 3] [1, 2]]

我想要这个

print(np.array([[1, 2, 3], [1, 2, np.nan]]))

[[  1.   2.   3.]
 [  1.   2.  nan]]

---

我可以通过循环来做到这一点,但我们都知道不受欢迎的循环是多少

def box_pir(l):
    lengths = [i for i in map(len, l)]
    shape = (len(l), max(lengths))
    a = np.full(shape, np.nan)
    for i, r in enumerate(l):
        a[i, :lengths[i]] = r
    return a

print(box_pir(l))

[[  1.   2.   3.]
 [  1.   2.  nan]]

---

我该如何以快速,矢量化的方式做到这一点？

---

定时

设置功能

%%cython
import numpy as np

def box_pir_cython(l):
    lengths = [len(item) for item in l]
    shape = (len(l), max(lengths))
    a = np.full(shape, np.nan)
    for i, r in enumerate(l):
        a[i, :lengths[i]] = r
    return a

---

def box_divikar(v):
    lens = np.array([len(item) for item in v])
    mask = lens[:,None] > np.arange(lens.max())
    out = np.full(mask.shape, np.nan)
    out[mask] = np.concatenate(v)
    return out

def box_hpaulj(LoL):
    return np.array(list(zip_longest(*LoL, fillvalue=np.nan))).T

def box_simon(LoL):
    max_len = len(max(LoL, key=len))
    return np.array([x + [np.nan]*(max_len-len(x)) for x in LoL])

def box_dawg(LoL):
    cols=len(max(LoL, key=len))
    rows=len(LoL)
    AoA=np.empty((rows,cols, ))
    AoA.fill(np.nan)
    for idx in range(rows):
        AoA[idx,0:len(LoL[idx])]=LoL[idx]
    return AoA

def box_pir(l):
    lengths = [len(item) for item in l]
    shape = (len(l), max(lengths))
    a = np.full(shape, np.nan)
    for i, r in enumerate(l):
        a[i, :lengths[i]] = r
    return a

def box_pandas(l):
    return pd.DataFrame(l).values

Answer 1

这似乎是一个接近的this question,填充与zeros而不是NaNs.有趣的方法发布在那里,并mine基于broadcasting和 boolean-indexing.那么,我只想修改我的帖子中的一行来解决这种情况,就像这样 -

def boolean_indexing(v, fillval=np.nan):
    lens = np.array([len(item) for item in v])
    mask = lens[:,None] > np.arange(lens.max())
    out = np.full(mask.shape,fillval)
    out[mask] = np.concatenate(v)
    return out

样品运行 -

In [32]: l
Out[32]: [[1, 2, 3], [1, 2], [3, 8, 9, 7, 3]]

In [33]: boolean_indexing(l)
Out[33]: 
array([[  1.,   2.,   3.,  nan,  nan],
       [  1.,   2.,  nan,  nan,  nan],
       [  3.,   8.,   9.,   7.,   3.]])

In [34]: boolean_indexing(l,-1)
Out[34]: 
array([[ 1,  2,  3, -1, -1],
       [ 1,  2, -1, -1, -1],
       [ 3,  8,  9,  7,  3]])

我已经在那里发布了一些运行结果,用于该问答的所有发布方法,这可能很有用.