两个设备故障之间的日期差异

Question

我正在尝试计算之间的天数failures。我想知道系列中每一天从最后一个failure位置过去的天数failure = 1。可能有1到1500个设备。

例如，我希望我的数据框看起来像这样（请从第二个代码块的url中提取数据。这只是较大数据框的简短示例。）：

date        device      failure      elapsed    
10/01/2015  S1F0KYCR    1            0           
10/07/2015  S1F0KYCR    1            7           
10/08/2015  S1F0KYCR    0            0           
10/09/2015  S1F0KYCR    0            0           
10/17/2015  S1F0KYCR    1            11          
10/31/2015  S1F0KYCR    0            0           
10/01/2015  S8KLM011    1            0           
10/02/2015  S8KLM011    1            2           
10/07/2015  S8KLM011    0            0
10/09/2015  S8KLM011    0            0
10/11/2015  S8KLM011    0            0
10/21/2015  S8KLM011    1            20 

样例代码：

编辑：请从下面的代码块中提取实际数据。以上示例数据是一个简短的示例。谢谢。

url = "https://raw.githubusercontent.com/dsdaveh/device-failure-analysis/master/device_failure.csv"

df = pd.read_csv(url, encoding = "ISO-8859-1")

df = df.sort_values(by = ['date', 'device'], ascending = True) #Sort by date and device
df['date'] = pd.to_datetime(df['date'],format='%Y/%m/%d') #format date to datetime

这是我遇到障碍的地方。但是，新列应包含自上次以来的天数failure，其中failure = 1。

test['date'] = 0
for i in test.index[1:]:
    if not test['failure'][i]:
        test['elapsed'][i] = test['elapsed'][i-1] + 1

我也尝试过

fails = df[df.failure==1]
fails.Dates = trues.index #need this because .diff() won't work on the index..
fails.Elapsed = trues.Dates.diff()

Answer 1

pandas.DataFrame.groupby与diff和结合使用apply：

import pandas as pd
import numpy as np

df['date'] = pd.to_datetime(df['date'])
s = df.groupby(['device', 'failure'])['date'].diff().dt.days.add(1)
s = s.fillna(0)
df['elapsed'] = np.where(df['failure'], s, 0)

输出：

         Date    Device  Failure  Elapsed
0  2015-10-01  S1F0KYCR        1      0.0
1  2015-10-07  S1F0KYCR        1      7.0
2  2015-10-08  S1F0KYCR        0      0.0
3  2015-10-09  S1F0KYCR        0      0.0
4  2015-10-17  S1F0KYCR        1     11.0
5  2015-10-31  S1F0KYCR        0      0.0
6  2015-10-01  S8KLM011        1      0.0
7  2015-10-02  S8KLM011        1      2.0
8  2015-10-07  S8KLM011        0      0.0
9  2015-10-09  S8KLM011        0      0.0
10 2015-10-11  S8KLM011        0      0.0
11 2015-10-21  S8KLM011        1     20.0

更新：

发现在OP中链接的实际数据不包含具有两个以上故障情况的设备，从而使最终结果全为零（即，从未发生过第二次故障，因此没有要计算的时间）。使用OP的原始代码段：

import pandas as pd

url = "http://aws-proserve-data-science.s3.amazonaws.com/device_failure.csv"

df = pd.read_csv(url, encoding = "ISO-8859-1")
df = df.sort_values(by = ['date', 'device'], ascending = True) 
df['date'] = pd.to_datetime(df['date'],format='%Y/%m/%d')

查找任何设备是否有超过1个故障：

df.groupby(['device'])['failure'].sum().gt(1).any()
# False

这实际上确认了所有的零df['elapsed']实际上是正确的答案:)

如果你调整你的数据位，它的产量经过与预期。

df.loc[6879, 'device'] = 'S1F0RRB1'
# Making two occurrence of failure for device S1F0RRB1

s = df.groupby(['device', 'failure'])['date'].diff().dt.days.add(1)
s = s.fillna(0)
df['elapsed'] = np.where(df['failure'], s, 0)
df['elapsed'].value_counts()
# 0.0    124493
# 3.0         1