timedelta csv pandas

Question

我有以下文件(df_SOF1.csv),它是100万条记录长

Location,Transport,Transport1,DateOccurred,CostCentre,D_Time,count
0,Lorry,Car,07/09/2012,0,0:00:00,2
1,Lorry,Car,11/09/2012,0,0:00:00,5
2,Lorry,Car,14/09/2012,0,0:00:00,30
3,Lorry,Car,14/09/2012,0,0:07:00,2
4,Lorry,Car,14/09/2012,0,0:29:00,1
5,Lorry,Car,14/09/2012,0,3:27:00,3
6,Lorry,Car,14/09/2012,0,3:28:00,4
7,Lorry,Car,21/09/2012,0,0:00:00,13
8,Lorry,Car,27/09/2012,0,0:00:00,8
9,Lorry,Car,28/09/2012,0,0:02:00,1
10,Train,Bus,03/09/2012,2073,7:49:00,1
11,Train,Bus,05/09/2012,2073,7:50:00,1
12,Train,Bus,06/09/2012,2073,7:52:00,1
13,Train,Bus,07/09/2012,2073,7:48:00,1
14,Train,Bus,08/09/2012,2073,7:55:00,1
15,Train,Bus,11/09/2012,2073,7:49:00,1
16,Train,Bus,12/09/2012,2073,7:52:00,1
17,Train,Bus,13/09/2012,2073,7:50:00,1
18,Train,Bus,14/09/2012,2073,7:54:00,1
19,Train,Bus,18/09/2012,2073,7:51:00,1
20,Train,Bus,19/09/2012,2073,7:50:00,1
21,Train,Bus,20/09/2012,2073,7:51:00,1
22,Train,Bus,21/09/2012,2073,7:52:00,1
23,Train,Bus,22/09/2012,2073,7:53:00,1
24,Train,Bus,23/09/2012,2073,7:49:00,1
25,Train,Bus,24/09/2012,2073,7:54:00,1
26,Train,Bus,25/09/2012,2073,7:55:00,1
27,Train,Bus,26/09/2012,2073,7:53:00,1
28,Train,Bus,27/09/2012,2073,7:55:00,1
29,Train,Bus,28/09/2012,2073,7:53:00,1
30,Train,Bus,29/09/2012,2073,7:56:00,1

我正在使用pandas来分析它我一直在尝试至少40个小时来找到一种方法来分组数据,我可以聚合时间列 D_Time

我已经加载了我创建数据帧所需的模块,请参阅下面DateOccured的索引

df_SOF1 = read_csv('/users/fabulous/documents/df_SOF1.csv', index_col=3, parse_dates=True) # read file from disk

我可以按任何列分组或遍历任何行,例如

df_SOF1.groupby('Location').sum()

但是我没有找到一种方法来总结并D_Time使用pandas 取出列的平均值.我已经阅读了20多篇关于timedeltas等的文章,但我仍然不是在大熊猫中如何做到这一点.

任何可以让我对D_Time列进行算术运算的解决方案都将受到赞赏.(即使它必须在熊猫之外完成).

我认为一种可能的解决方案是将D_Time列更改为秒. _ __ _ __ _ __ _ __ _ __ _ __ _ __ _ __ _ __ _ __ _ __ _2012/11/01 I运行以下命令对30个项目以上

df_SOF1.groupby('Transport').agg({'D_Time':sum})

D_Time

交通
卡车0:00:000:00:000:00:000:07:000:29:003:27:003:28 ...火车7:49:007:50:007:52:007:48:007 :55:007:49:007:52 ..

它似乎是将物理上的值相加而不是给出数值总和(比如添加字符串)

干杯

Answer 1

我没有在 pandas 中找到任何关于 deltatime 的提及，并且 datetime 模块有一个，因此将 D_Time 转换为秒不是一个坏主意：

def seconds(time_str):
    end_time = datetime.datetime.strptime(time_str,'%H:%M:%S')
    delta = end_time - datetime.datetime.strptime('0:0:0','%H:%M:%S')
    return delta.total_seconds()


df_SOF1.D_Time = df_SOF1.D_Time.apply(seconds)

结果 ：

>>> df_SOF1.groupby('CostCentre').sum()
            Location  D_Time  count
CostCentre                         
0                 45   27180     69
2073             420  594660     21

将 datetime.datetime.strptime('0:0:0','%H:%M:%S') 移动到全局命名空间可以减少执行时间：

timeit.timeit("sec('01:01:01')", setup="from __main__ import sec",
              number=10000)
1.025843858718872

timeit.timeit("seconds('01:01:01')", setup="from __main__ import seconds",
              number=10000)
0.6128969192504883