更快地实施pandas应用功能

Question

我有一个pandas dataFrame我想检查一列是否contained在另一列中.

假设:

df = DataFrame({'A': ['some text here', 'another text', 'and this'], 
                'B': ['some', 'somethin', 'this']})

我想检查是否df.B[0]在df.A[0],df.B[1]是否在df.A[1]等

目前的做法

我有以下apply功能实现

df.apply(lambda x: x[1] in x[0], axis=1)

结果是Series的[True, False, True]

这很好,但对于我dataFrame shape(它是数百万)它需要很长时间.
是否有更好(即更快)的植入？

Unsuccesfull方法

我尝试了这种pandas.Series.str.contains方法,但它只能为模式采用字符串.

df['A'].str.contains(df['B'], regex=False)

Answer 1

使用np.vectorize- 绕过apply开销,所以应该更快一点.

v = np.vectorize(lambda x, y: y in x)

v(df.A, df.B)
array([ True, False,  True], dtype=bool)

这是一个时间比较 -

df = pd.concat([df] * 10000)

%timeit df.apply(lambda x: x[1] in x[0], axis=1)
1 loop, best of 3: 1.32 s per loop

%timeit v(df.A, df.B)
100 loops, best of 3: 5.55 ms per loop

# Psidom's answer
%timeit [b in a for a, b in zip(df.A, df.B)]
100 loops, best of 3: 3.34 ms per loop

这两个都是非常有竞争力

编辑,为Wen和Max的答案添加时间 -

# Wen's answer
%timeit df.A.replace(dict(zip(df.B.tolist(),[np.nan]*len(df))),regex=True).isnull()
10 loops, best of 3: 49.1 ms per loop

# MaxU's answer
%timeit df['A'].str.split(expand=True).eq(df['B'], axis=0).any(1)
10 loops, best of 3: 87.8 ms per loop

Answer 2

尝试zip,它apply在这种情况下明显更快:

df = pd.concat([df] * 10000)
df.head()
#                A         B
#0  some text here      some
#1    another text  somethin
#2        and this      this
#0  some text here      some
#1    another text  somethin

%timeit df.apply(lambda x: x[1] in x[0], axis=1)
# 1 loop, best of 3: 697 ms per loop

%timeit [b in a for a, b in zip(df.A, df.B)]
# 100 loops, best of 3: 3.53 ms per loop

# @coldspeed's np.vectorize solution
%timeit v(df.A, df.B)
# 100 loops, best of 3: 4.18 ms per loop