按键更新pandas DataFrame

Question

我有一个历史股票交易的数据框.框架包含['ticker','date','cusip','profit','security_type']等列.原来:

trades['cusip'] = np.nan
trades['security_type'] = np.nan

我有历史配置文件,我可以加载到具有['ticker','cusip','date','name','security_type','primary_exchange']等列的框架中.

我想用配置中的cusip和security_type更新交易框架,但仅限于股票代码和日期匹配的地方.

我以为我可以这样做:

pd.merge(trades, config, on=['ticker', 'date'], how='left')

但是这不会更新列,它只是将配置列添加到交易中.

以下是有效的,但我认为必须有更好的方法.如果没有,我可能会在熊猫之外做.

for date in trades['date'].unique():
    config = get_config_file_as_df(date)
    ## config['date'] == date
    for ticker in trades['ticker'][trades['date'] == date]:
        trades['cusip'][ 
                           (trades['ticker'] == ticker)
                         & (trades['date']   == date)
                       ] \
            = config['cusip'][config['ticker'] == ticker].values[0]

        trades['security_type'][ 
                           (trades['ticker'] == ticker)
                         & (trades['date']   == date)
                       ] \
            = config['security_type'][config['ticker'] == ticker].values[0]

Answer 1

假设你有这个设置:

import pandas as pd
import numpy as np
import datetime as DT

nan = np.nan

trades = pd.DataFrame({'ticker' : ['IBM', 'MSFT', 'GOOG', 'AAPL'],
                       'date' : pd.date_range('1/1/2000', periods = 4), 
                       'cusip' : [nan, nan, 100, nan]
                       })
trades = trades.set_index(['ticker', 'date'])
print(trades)
#                    cusip
# ticker date             
# IBM    2000-01-01    NaN
# MSFT   2000-01-02    NaN
# GOOG   2000-01-03    100  # <-- We do not want to overwrite this
# AAPL   2000-01-04    NaN

config = pd.DataFrame({'ticker' : ['IBM', 'MSFT', 'GOOG', 'AAPL'],
                       'date' : pd.date_range('1/1/2000', periods = 4),
                       'cusip' : [1,2,3,nan]})
config = config.set_index(['ticker', 'date'])

# Let's permute the index to show `DataFrame.update` correctly matches rows based on the index, not on the order of the rows.
new_index = sorted(config.index)
config = config.reindex(new_index)    
print(config)
#                    cusip
# ticker date             
# AAPL   2000-01-04    NaN
# GOOG   2000-01-03      3
# IBM    2000-01-01      1
# MSFT   2000-01-02      2

然后你就可以在更新NaN值trades与值从config使用DataFrame.update方法.请注意,DataFrame.update根据索引匹配行(这就是set_index上面调用的原因).

trades.update(config, join = 'left', overwrite = False)
print(trades)

#                    cusip
# ticker date             
# IBM    2000-01-01      1
# MSFT   2000-01-02      2
# GOOG   2000-01-03    100 # If overwrite = True, then 100 is overwritten by 3.
# AAPL   2000-01-04    NaN