按行快速vector化data.frames列表的合并

大多数关于在SO上列表中合并data.frame的问题并不完全涉及到我想要在这里得到的东西,而是随意地certificate我错了。

我有一个data.frames的列表。 我想“行”到另一个data.frame逐行。 实质上,所有第一行形成一个dataframe,第二行第二个dataframe,等等。 结果将是一个长度与我的原始data.frame(s)行数相同的列表。 到目前为止,data.frames的维度是相同的。

这里有一些数据要玩。

sample.list <- list(data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE))) 

下面是我用良好的ol循环提出的。

 #solution 1 my.list <- vector("list", nrow(sample.list[[1]])) for (i in 1:nrow(sample.list[[1]])) { for (j in 1:length(sample.list)) { my.list[[i]] <- rbind(my.list[[i]], sample.list[[j]][i, ]) } } #solution 2 (so far my favorite) sample.list2 <- do.call("rbind", sample.list) my.list2 <- vector("list", nrow(sample.list[[1]])) for (i in 1:nrow(sample.list[[1]])) { my.list2[[i]] <- sample.list2[seq(from = i, to = nrow(sample.list2), by = nrow(sample.list[[1]])), ] } 

这可以通过使用vector化来改善,而不会有太多的脑筋? 当然,正确的答案将包含一段代码。 “是”作为答案不算。

编辑

 #solution 3 (a variant of solution 2 above) ind <- rep(1:nrow(sample.list[[1]]), times = length(sample.list)) my.list3 <- split(x = sample.list2, f = ind) 

标杆

我已经使我的列表更大,每个data.frame更多的行。 我已经对以下结果进行了基准testing:

 #solution 1 system.time(for (i in 1:nrow(sample.list[[1]])) { for (j in 1:length(sample.list)) { my.list[[i]] <- rbind(my.list[[i]], sample.list[[j]][i, ]) } }) user system elapsed 80.989 0.004 81.210 # solution 2 system.time(for (i in 1:nrow(sample.list[[1]])) { my.list2[[i]] <- sample.list2[seq(from = i, to = nrow(sample.list2), by = nrow(sample.list[[1]])), ] }) user system elapsed 0.957 0.160 1.126 # solution 3 system.time(split(x = sample.list2, f = ind)) user system elapsed 1.104 0.204 1.332 # solution Gabor system.time(lapply(1:nr, bind.ith.rows)) user system elapsed 0.484 0.000 0.485 # solution ncray system.time(alply(do.call("cbind",sample.list), 1, .fun=matrix, ncol=ncol(sample.list[[1]]), byrow=TRUE, dimnames=list(1:length(sample.list),names(sample.list[[1]])))) user system elapsed 11.296 0.016 11.365 

尝试这个:

 bind.ith.rows <- function(i) do.call(rbind, lapply(sample.list, "[", i, TRUE)) nr <- nrow(sample.list[[1]]) lapply(1:nr, bind.ith.rows) 

一些解决scheme将使这个使用data.table更快

编辑 – 更大的数据集显示data.table更多。

 # here are some sample data sample.list <- replicate(10000, data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), simplify = F) 

Gabor的快速解决scheme:

 # Solution Gabor bind.ith.rows <- function(i) do.call(rbind, lapply(sample.list, "[", i, TRUE)) nr <- nrow(sample.list[[1]]) system.time(rowbound <- lapply(1:nr, bind.ith.rows)) ## user system elapsed ## 25.87 0.01 25.92 

即使使用data.frames,data.table函数rbindlist也会更快)

 library(data.table) fastbind.ith.rows <- function(i) rbindlist(lapply(sample.list, "[", i, TRUE)) system.time(fastbound <- lapply(1:nr, fastbind.ith.rows)) ## user system elapsed ## 13.89 0.00 13.89 

一个data.table解决scheme

这是一个使用data.tables的解决scheme – 它是类固醇的split解决scheme。

 # data.table solution system.time({ # change each element of sample.list to a data.table (and data.frame) this # is done instaneously by reference invisible(lapply(sample.list, setattr, name = "class", value = c("data.table", "data.frame"))) # combine into a big data set bigdata <- rbindlist(sample.list) # add a row index column (by refere3nce) index <- as.character(seq_len(nr)) bigdata[, `:=`(rowid, index)] # set the key for binary searches setkey(bigdata, rowid) # split on this - dt_list <- lapply(index, function(i, j, x) x[i = J(i)], x = bigdata) # if you want to drop the `row id` column invisible(lapply(dt_list, function(x) set(x, j = "rowid", value = NULL))) # if you really don't want them to be data.tables run this line # invisible(lapply(dt_list, setattr,name = 'class', value = # c('data.frame'))) }) ################################ ## user system elapsed ## ## 0.08 0.00 0.08 ## ################################ 

data.table真棒!

警告用户与rbindlist

rbindlist速度很快,因为它不执行do.call(rbind,....)将会执行的检查。 例如,假定任何因子列与列表的第一个元素具有相同的级别。

这是我尝试与plyr,但我喜欢格洛腾迪克的方法:

 library(plyr) alply(do.call("cbind",sample.list), 1, .fun=matrix, ncol=ncol(sample.list[[1]]), byrow=TRUE, dimnames=list(1:length(sample.list), names(sample.list[[1]]) ))