计算 R data.table 中多个问题的分位数，同时计算多个列

Question

DT = data.table(x=rep(c("b","a","c"),each=3), y=c(1,3,6), v=1:9)

# Desired output
rbind(cbind(id = "v", DT[x == "a", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("v")]),
      cbind(id = "y", DT[x == "a", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("y")]),

      cbind(id = "v", DT[x == "b", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("v")]),
      cbind(id = "y", DT[x == "b", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("y")]),
      cbind(id = "v", DT[x == "c", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("v")]),
      cbind(id = "y", DT[x == "c", as.list(quantile(.SD, prob = c(0.05, .5, 0.95), na.rm = T)), by = x, .SDcols = c("y")])
)
#    id x  5% 50% 95%
# 1:  v a 4.1   5 5.9
# 2:  y a 1.2   3 5.7
# 3:  v b 1.1   2 2.9
# 4:  y b 1.2   3 5.7
# 5:  v c 7.1   8 8.9
# 6:  y c 1.2   3 5.7

如何使用 data.table （几 GB 内存）在非常大的数据集上有效地实现上述输出？我已经尝试过，但这不是我想要的

# not right, want all 3 percentiles on the same row, for x and then y:
out <- DT[ , lapply(.SD, quantile, prob = c(0.05, .5, 0.95), na.rm = T), .SDcols = c("v", "y"), keyby = "x"]
out

那么我怎样才能得到上面我想要的输出，但 id 分布在各列上，这样它就变成了 3 x 6 data.table。例如，列 v5% v50% v95% y5% y50% y95% 包含 3 行。

Answer 1

您可以使用以下方法melt/dcast来实现此目的：

dcast(melt(out[, p := rep(paste0(c(5, 50, 95), "%"), 3)], 
           c("p", "x"), 
           variable.name = "id"), 
      id + x ~ ...)[order(x, id)]
#    id x  5% 50% 95%
# 1:  v a 4.1   5 5.9
# 2:  y a 1.2   3 5.7
# 3:  v b 1.1   2 2.9
# 4:  y b 1.2   3 5.7
# 5:  v c 7.1   8 8.9
# 6:  y c 1.2   3 5.7

另一种没有中间结果的选择；

melt(DT[, v := as.numeric(v)], 
     "x",
     c("v", "y"),
     variable.name = "id")[, as.list(quantile(value, 
                                              prob = c(.05, .5, .95))), 
                           .(x, id)][order(x, id)]
#    x id  5% 50% 95%
# 1: a  v 4.1   5 5.9
# 2: a  y 1.2   3 5.7
# 3: b  v 1.1   2 2.9
# 4: b  y 1.2   3 5.7
# 5: c  v 7.1   8 8.9
# 6: c  y 1.2   3 5.7

笔记。我将列转换v为numeric(from int) 以避免来自 的令人讨厌的警告melt。