Python：如何根据第一列中的值将 Pandas DataFrame 拆分为子集？

Question

我有一个很大的实验日志文件 (.txt)（最多包含 100 000 个条目），其结构如下：

ROUTINE    TEMPERATURE    VOLTAGE    WAVELENGTH
_______________________________________________
CHANGE T   75             0          560
CHANGE T   80             0          560
CHANGE T   85             0          560
CHANGE T   90             0          560
OSL        75             20         570
OSL        75             20         580
OSL        75             20         590
OSL        75             20         600
CHANGE T   75             0          560
CHANGE T   80             0          560
CHANGE T   85             0          560
CHANGE T   90             0          560

我使用日志文件加载到蟒蛇read_table从大熊猫。我想根据第一列的值将结果数据帧分成更小的数据帧。所以结果应该是这样的：

**DATAFRAME 1:**    
CHANGE T   75             0          560
CHANGE T   80             0          560
CHANGE T   85             0          560
CHANGE T   90             0          560

**DATAFRAME 2:** 
OSL        75             20         570
OSL        75             20         580
OSL        75             20         590
OSL        75             20         600

**DATAFRAME 3:** 
CHANGE T   75             0          560
CHANGE T   80             0          560
CHANGE T   85             0          560
CHANGE T   90             0          560

首先，我尝试使用第一列值发生变化的索引拆分它们：

indexSplit = [] # list containing the boundry indices

prevRoutine = log['ROUTINE'][0] # log is the complete dataframe
i = 1
while i < len(log):
        if prevRoutine != log['ROUTINE'][i]:
            indexSplit.append(i)
        prevRoutine = log['ROUTINE'][i]

但是，考虑到日志文件的大小，以这种方式（显然）需要花费大量时间。我想知道是否有一种优雅的方法可以用熊猫来做到这一点？我一直遇到的问题是第一列的值用于多个系列。我总是将数据帧 1和数据帧 3 合二为一。

Answer 1

您可以使用list comprehensionwhere 循环groupby对象groups并由s. 有比较ne（相同!=但更快）shifted 列和通过cumsumget 输出：

s = df['ROUTINE'].ne(df['ROUTINE'].shift()).cumsum()
print (s)
0     1
1     1
2     1
3     1
4     2
5     2
6     2
7     2
8     3
9     3
10    3
11    3
Name: ROUTINE, dtype: int32

dfs = [g for i,g in df.groupby(df['ROUTINE'].ne(df['ROUTINE'].shift()).cumsum())]
print (dfs)
[    ROUTINE  TEMPERATURE  VOLTAGE  WAVELENGTH
0  CHANGE T           75        0         560
1  CHANGE T           80        0         560
2  CHANGE T           85        0         560
3  CHANGE T           90        0         560,   ROUTINE  TEMPERATURE  VOLTAGE  WAVELENGTH
4     OSL           75       20         570
5     OSL           75       20         580
6     OSL           75       20         590
7     OSL           75       20         600,      ROUTINE  TEMPERATURE  VOLTAGE  WAVELENGTH
8   CHANGE T           75        0         560
9   CHANGE T           80        0         560
10  CHANGE T           85        0         560
11  CHANGE T           90        0         560]

print (dfs[0])
    ROUTINE  TEMPERATURE  VOLTAGE  WAVELENGTH
0  CHANGE T           75        0         560
1  CHANGE T           80        0         560
2  CHANGE T           85        0         560
3  CHANGE T           90        0         560

print (dfs[1])
  ROUTINE  TEMPERATURE  VOLTAGE  WAVELENGTH
4     OSL           75       20         570
5     OSL           75       20         580
6     OSL           75       20         590
7     OSL           75       20         600

print (dfs[2])
     ROUTINE  TEMPERATURE  VOLTAGE  WAVELENGTH
8   CHANGE T           75        0         560
9   CHANGE T           80        0         560
10  CHANGE T           85        0         560
11  CHANGE T           90        0         560

解决方案很复杂，因为如果groupby用于第一列只得到 2 组：

dfs = [g for i,g in df.groupby('ROUTINE')]
print (dfs)
[     ROUTINE  TEMPERATURE  VOLTAGE  WAVELENGTH
0   CHANGE T           75        0         560
1   CHANGE T           80        0         560
2   CHANGE T           85        0         560
3   CHANGE T           90        0         560
8   CHANGE T           75        0         560
9   CHANGE T           80        0         560
10  CHANGE T           85        0         560
11  CHANGE T           90        0         560,   ROUTINE  TEMPERATURE  VOLTAGE  WAVELENGTH
4     OSL           75       20         570
5     OSL           75       20         580
6     OSL           75       20         590
7     OSL           75       20         600]