小编kos*_*tas的帖子

优化pandas中运行时的过滤功能

def filter_data(df, raw_col,threshold,filt_col):
    df['pct'] = None
    df[filt_col] = None
    df[filt_col][0] = df[raw_col][0]
    max_val = df[raw_col][0]
    for i in range(1,len(df)):
        df['pct'][i] = (df[raw_col][i] - max_val)*1.0 / max_val
        if abs(df['pct'][i]) < threshold:
            df[filt_col][i] = None
        else:
            df[filt_col][i] = df[raw_col][i]
            max_val = df[raw_col][i]
    df = df.dropna(axis=0, how='any').reset_index()
    return df


from random import randint
some_lst = [randint(50, 100) for i in range(0,50)]
some_df = pd.DataFrame({'raw_col':some_lst})
some_df_filt = filter_data(some_df,'raw_col',0.01,'raw_col_filt')
Run Code Online (Sandbox Code Playgroud)

创建新列(filt_col)的目标是使用以下逻辑删除数字列(raw_col)中的记录; 如果两个相邻行之间的变化率小于阈值,则移除后者.它有效,但在运行时间方面效率很低.有关如何优化它的任何提示?

python filter pandas

5
推荐指数
1
解决办法
132
查看次数

标签 统计

filter ×1

pandas ×1

python ×1