pandas - groupby和过滤连续值

Question

我有这个数据帧df:

U,Datetime
01,2015-01-01 20:00:00
01,2015-02-01 20:05:00
01,2015-04-01 21:00:00
01,2015-05-01 22:00:00
01,2015-07-01 22:05:00
02,2015-08-01 20:00:00
02,2015-09-01 21:00:00
02,2014-01-01 23:00:00
02,2014-02-01 22:05:00
02,2015-01-01 20:00:00
02,2014-03-01 21:00:00
03,2015-10-01 20:00:00
03,2015-11-01 21:00:00
03,2015-12-01 23:00:00
03,2015-01-01 22:05:00
03,2015-02-01 20:00:00
03,2015-05-01 21:00:00
03,2014-01-01 20:00:00
03,2014-02-01 21:00:00

通过由U与一个Datetime对象.我想做的是过滤U几个月/年至少连续三次出现的值.到目前为止,我已经通过分组U,year并month为:

m = df.groupby(['U',df.index.year,df.index.month]).size()

获得:

U          
1  2015  1     1
         2     1
         4     1
         5     1
         7     1
2  2014  1     1
         2     1
         3     1
   2015  1     1
         8     1
         9     1
3  2014  1     1
         2     1
   2015  1     1
         2     1
         5     1
         10    1
         11    1
         12    1

第三栏与不同月/年的事件有关.在这种情况下,只有几个月/年的U值02和03包含至少三个连续值.现在我无法弄清楚如何选择这些用户并将它们列在列表中,或者只是将它们保留在原始数据帧中df并丢弃其他用户.我也尝试过:

g = m.groupby(level=[0,1]).diff()

但我无法得到任何有用的信息.

Answer 1

最后我可以想出解决方案:)。

\n\n

为了让您了解自定义函数的工作原理，只需从先前的值中减去月份的值，结果当然应该是one，并且这应该发生两次，例如，如果您有一个数字列表[5 , 6 , 7]，那么7 - 6 = 1和6 - 5 = 1，1这里出现了两次所以条件已经满足

\n\n

In [80]:\ndf.reset_index(inplace=True)\n\nIn [281]:\ndf['month'] = df.Datetime.dt.month\ndf['year'] = df.Datetime.dt.year\ndf\nOut[281]:\n            Datetime    U   month   year\n0   2015-01-01 20:00:00 1   1       2015\n1   2015-02-01 20:05:00 1   2       2015\n2   2015-04-01 21:00:00 1   4       2015\n3   2015-05-01 22:00:00 1   5       2015\n4   2015-07-01 22:05:00 1   7       2015\n5   2015-08-01 20:00:00 2   8       2015\n6   2015-09-01 21:00:00 2   9       2015\n7   2014-01-01 23:00:00 2   1       2014\n8   2014-02-01 22:05:00 2   2       2014\n9   2015-01-01 20:00:00 2   1       2015\n10  2014-03-01 21:00:00 2   3       2014\n11  2015-10-01 20:00:00 3   10      2015\n12  2015-11-01 21:00:00 3   11      2015\n13  2015-12-01 23:00:00 3   12      2015\n14  2015-01-01 22:05:00 3   1       2015\n15  2015-02-01 20:00:00 3   2       2015\n16  2015-05-01 21:00:00 3   5       2015\n17  2014-01-01 20:00:00 3   1       2014\n18  2014-02-01 21:00:00 3   2       2014\n\nIn [284]:\ng = df.groupby([df['U'] , df.year])\n\nIn [86]:\nres = g.filter(lambda x : is_at_least_three_consec(x['month'].diff().values.tolist()))\nres\nOut[86]:\n      Datetime          U   month   year\n7   2014-01-01 23:00:00 2   1       2014\n8   2014-02-01 22:05:00 2   2       2014\n10  2014-03-01 21:00:00 2   3       2014\n11  2015-10-01 20:00:00 3   10      2015\n12  2015-11-01 21:00:00 3   11      2015\n13  2015-12-01 23:00:00 3   12      2015\n14  2015-01-01 22:05:00 3   1       2015\n15  2015-02-01 20:00:00 3   2       2015\n16  2015-05-01 21:00:00 3   5       2015\n

\n\n

如果你想查看自定义函数的结果

\n\n

In [84]:\nres = g['month'].agg(lambda x : is_at_least_three_consec(x.diff().values.tolist()))\nres\nOut[84]:\nU  year\n1  2015    False\n2  2014     True\n   2015    False\n3  2014    False\n   2015     True\nName: month, dtype: bool\n

\n\n

这就是自定义函数的实现方式

\n\n

In [53]:    \ndef is_at_least_three_consec(month_diff):\n    consec_count = 0\n    #print(month_diff)\n    for index , val in enumerate(month_diff):\n        if index != 0 and val == 1:\n                consec_count += 1\n                if consec_count == 2:\n                    return True\n        else:\n            consec_count = 0\n\xe2\x80\x8b\n    return False\n

\n