如何对使用其自身输出的滞后值的函数进行矢量化？
ves*_*and 7 python numpy vectorization ipython pandas
对于这个问题的糟糕措辞,我感到很遗憾,但这是我能做的最好的事情.我确切地知道我想要什么,但不知道如何要求它.
以下是一个示例演示的逻辑:
采用值1或0的两个条件触发一个也取值为1或0的信号.条件A触发信号(如果A = 1则信号= 1,否则信号= 0)无论如何.条件B不触发信号,但是如果条件B在条件A先前已经触发信号之后保持等于1,则信号保持触发.仅在A和B都返回到0之后,信号才返回到0.
1.输入:
2.期望的输出(signal_d)并确认for循环可以解决它(signal_l):
3.我尝试使用numpy.where():
4.可重复的代码片段:
# Settings
import numpy as np
import pandas as pd
import datetime
# Data frame with input and desired output i column signal_d
df = pd.DataFrame({'condition_A':list('00001100000110'),
'condition_B':list('01110011111000'),
'signal_d':list('00001111111110')})
colnames = list(df)
df[colnames] = df[colnames].apply(pd.to_numeric)
datelist = pd.date_range(pd.datetime.today().strftime('%Y-%m-%d'), periods=14).tolist()
df['dates'] = datelist
df = df.set_index(['dates'])
# Solution using a for loop with nested ifs in column signal_l
df['signal_l'] = df['condition_A'].copy(deep = True)
i=0
for observations in df['signal_l']:
if df.ix[i,'condition_A'] == 1:
df.ix[i,'signal_l'] = 1
else:
# Signal previously triggered by condition_A
# AND kept "alive" by condition_B:
if df.ix[i - 1,'signal_l'] & df.ix[i,'condition_B'] == 1:
df.ix[i,'signal_l'] = 1
else:
df.ix[i,'signal_l'] = 0
i = i + 1
# My attempt with np.where in column signal_v1
df['Signal_v1'] = df['condition_A'].copy()
df['Signal_v1'] = np.where(df.condition_A == 1, 1, np.where( (df.shift(1).Signal_v1 == 1) & (df.condition_B == 1), 1, 0))
print(df)
这是非常简单的使用带有滞后值的for循环和嵌套if句子,但我无法用像矢量化函数来解决它numpy.where().而且我知道对于更大的数据帧来说这会更快.
谢谢你的任何建议!
我认为没有一种方法可以比 Python 循环更快地矢量化此操作。（至少，如果你只想坚持使用 Python、pandas 和 numpy，则不需要。）
\n\n但是，您可以通过简化代码来提高此操作的性能。您的实现使用if语句和大量 DataFrame 索引。这些都是成本相对较高的操作。
这是对脚本的修改，其中包括两个函数：add_signal_l(df)和add_lagged(df)。第一个是您的代码，只是包含在一个函数中。第二个使用更简单的函数来实现相同的结果——仍然是 Python 循环，但它使用 numpy 数组和按位运算符。
import numpy as np\nimport pandas as pd\nimport datetime\n\n#-----------------------------------------------------------------------\n# Create the test DataFrame\n\n# Data frame with input and desired output i column signal_d\ndf = pd.DataFrame({\'condition_A\':list(\'00001100000110\'),\n \'condition_B\':list(\'01110011111000\'),\n \'signal_d\':list(\'00001111111110\')})\n\ncolnames = list(df)\ndf[colnames] = df[colnames].apply(pd.to_numeric)\ndatelist = pd.date_range(pd.datetime.today().strftime(\'%Y-%m-%d\'), periods=14).tolist()\ndf[\'dates\'] = datelist\ndf = df.set_index([\'dates\']) \n#-----------------------------------------------------------------------\n\ndef add_signal_l(df):\n # Solution using a for loop with nested ifs in column signal_l\n df[\'signal_l\'] = df[\'condition_A\'].copy(deep = True)\n i=0\n for observations in df[\'signal_l\']:\n if df.ix[i,\'condition_A\'] == 1:\n df.ix[i,\'signal_l\'] = 1\n else:\n # Signal previously triggered by condition_A\n # AND kept "alive" by condition_B: \n if df.ix[i - 1,\'signal_l\'] & df.ix[i,\'condition_B\'] == 1:\n df.ix[i,\'signal_l\'] = 1\n else:\n df.ix[i,\'signal_l\'] = 0 \n i = i + 1\n\ndef compute_lagged_signal(a, b):\n x = np.empty_like(a)\n x[0] = a[0]\n for i in range(1, len(a)):\n x[i] = a[i] | (x[i-1] & b[i])\n return x\n\ndef add_lagged(df):\n df[\'lagged\'] = compute_lagged_signal(df[\'condition_A\'].values, df[\'condition_B\'].values)\n下面是在 IPython 会话中运行的两个函数的时间比较：
\n\nIn [85]: df\nOut[85]: \n condition_A condition_B signal_d\ndates \n2017-06-09 0 0 0\n2017-06-10 0 1 0\n2017-06-11 0 1 0\n2017-06-12 0 1 0\n2017-06-13 1 0 1\n2017-06-14 1 0 1\n2017-06-15 0 1 1\n2017-06-16 0 1 1\n2017-06-17 0 1 1\n2017-06-18 0 1 1\n2017-06-19 0 1 1\n2017-06-20 1 0 1\n2017-06-21 1 0 1\n2017-06-22 0 0 0\n\nIn [86]: %timeit add_signal_l(df)\n8.45 ms \xc2\xb1 177 \xc2\xb5s per loop (mean \xc2\xb1 std. dev. of 7 runs, 100 loops each)\n\nIn [87]: %timeit add_lagged(df)\n137 \xc2\xb5s \xc2\xb1 581 ns per loop (mean \xc2\xb1 std. dev. of 7 runs, 10000 loops each)\n正如您所看到的，add_lagged(df)速度要快得多。