将字符向量拆分为数据框中的新行的最快方法

Question

我不确定如何在搜索时正确说出这一点,很抱歉,如果这有一个简单的答案.

我有58个数据帧,每行约25,000行.csv's.他们看起来像这样:

Probe.Id     Gene.Id             Score.d
1418126_at   6352                28.52578
145119_a_at  2192                24.87866
1423477_at   NA                  24.43532
1434193_at   100506144///9204    6.22395

理想情况下,我想在"///"处拆分ID并将它们放在新行上.像这样:

Probe.Id     Gene.Id             Score.d
1418126_at   6352                28.52578
145119_a_at  2192                24.87866
1423477_at   NA                  24.43532
1434193_at   100506144           6.22395
1434193_at   9204                6.22395

使用strsplit允许我将Gene.Id作为一个字符向量列表,但是一旦我有了这个,我不知道最有效的方法是使用另一个正确的值来获取每个单独的id在他们自己的行上列.理想情况下,我不想只循环25,000行.

如果有人知道正确的方法,我会非常感激.

编辑:我应该补充说,有一个复杂的因素,有些行有像这样的ID:

333932///126961///653604///8350///8354///8355///8356///8968///8352///8358///835??1///8353///8357" 

我不知道连续的最大数量是多少.

Answer 1

编辑: OP评论后的新解决方案.非常直接的使用data.table:

df <- structure(list(Probe.Id = c("1418126_at", "145119_a_at", "1423477_at", 
        "1434193_at", "100_at"), Gene.Id = c("6352", "2192", NA, 
        "100506144///9204", "100506144///100506146///100506148///100506150"), 
         Score.d = c(28.52578, 24.87866, 24.43532, 6.22395, 6.22395)), 
        .Names = c("Probe.Id", "Gene.Id", "Score.d"), row.names = c(NA, 5L), 
        class = "data.frame")

require(data.table)
dt <- data.table(df)
dt.out <- dt[, list(Probe.Id = Probe.Id, 
          Gene.Id = unlist(strsplit(Gene.Id, "///")), 
          Score.d = Score.d), by=1:nrow(dt)]

> dt.out

#    nrow    Probe.Id   Gene.Id  Score.d
# 1:    1  1418126_at      6352 28.52578
# 2:    2 145119_a_at      2192 24.87866
# 3:    3  1423477_at        NA 24.43532
# 4:    4  1434193_at 100506144  6.22395
# 5:    4  1434193_at      9204  6.22395
# 6:    5      100_at 100506144  6.22395
# 7:    5      100_at 100506146  6.22395
# 8:    5      100_at 100506148  6.22395
# 9:    5      100_at 100506150  6.22395

如果是固定模式,您可以添加fixed = TRUE到strsplit表达式以进一步加速///.

替代再次使用data.table.考虑到这strsplit是一个矢量化操作,并且在整个Gene.Id列上运行它会比一次运行一行快得多(尽管data.table运行速度非常快,你可以通过拆分前面的代码来获得更多的加速分为两步:

# first split using strsplit (data.table can hold list in its columns!!)
dt[, Gene.Id_split := strsplit(dt$Gene.Id, "///", fixed=TRUE)]
# then just unlist them
dt.2 <- dt[, list(Probe.Id = Probe.Id, 
                  Gene.Id = unlist(Gene.Id_split), 
                  Score.d = Score.d), by = 1:nrow(dt)]

我刚刚复制了data.table这个例子中的显示,直到我得到了295245行.然后我运行了一个基准测试rbenchmark:

# first function
DT1 <- function() {
    dt.1 <- dt[, list(Probe.Id = Probe.Id, 
             Gene.Id = unlist(strsplit(Gene.Id, "///", fixed = TRUE)), 
             Score.d = Score.d), by=1:nrow(dt)]
}

# expected to be faster function
DT2 <- function() {
    dt[, Gene.Id_split := strsplit(dt$Gene.Id, "///", fixed=TRUE)]
    # then just unlist them
    dt.2 <- dt[, list(Probe.Id = Probe.Id, Gene.Id = unlist(Gene.Id_split), Score.d = Score.d), by = 1:nrow(dt)]
}

require(rbenchmark)
benchmark(DT1(), DT2(), replications=10, order="elapsed")

#    test replications elapsed relative user.self sys.self
# 2 DT2()           10  15.708    1.000    14.390    0.391
# 1 DT1()           10  24.957    1.589    23.723    0.436

对于此示例,您的速度提高约1.6倍.但这取决于条目的数量///.希望这可以帮助.

OLD解决方案:(用于连续性)

一种方法是:1)find the positions发生这种///情况,2)extract,3)duplicate,4)sub和5)combine它们.

df <- structure(list(Probe.Id = structure(c(1L, 4L, 2L, 3L), 
         .Label = c("1418126_at", "1423477_at", "1434193_at", "145119_a_at"), 
         class = "factor"), Gene.Id = structure(c(3L, 2L, NA, 1L), 
         .Label = c("100506144///9204", "2192", "6352"), class = "factor"), 
         Score.d = c(28.52578, 24.87866, 24.43532, 6.22395)), 
         .Names = c("Probe.Id", "Gene.Id", "Score.d"), 
         class = "data.frame", row.names = c(NA, -4L))

# 1) get the positions of "///"
idx <- grepl("[/]{3}", df$Gene.Id)

# 2) create 3 data.frames
df1 <- df[!idx, ] # don't touch this.
df2 <- df[idx, ] # we need to work on this

# 3) duplicate
df3 <- df2 # duplicate it.

4) sub    
df2$Gene.Id <- sub("[/]{3}.*$", "", df2$Gene.Id) # replace the end
df3$Gene.Id <- sub("^.*[/]{3}", "", df3$Gene.Id) # replace the beginning

# 5) combine/put them back
df.out <- rbind(df1, df2, df3)

# if necessary sort them here.