读取多个文件并根据用户input计算平均值

我想在R中写一个函数,它需要3个input:

  1. 目录
  2. 污染物
  3. ID

我的电脑上有一个目录,里面装满了CSV文件,比如300多个。这个函数的function如下:

pollutantmean <- function(directory, pollutant, id = 1:332) { ## 'directory' is a character vector of length 1 indicating ## the location of the CSV files ## 'pollutant' is a character vector of length 1 indicating ## the name of the pollutant for which we will calculate the ## mean; either "sulfate" or "nitrate". ## 'id' is an integer vector indicating the monitor ID numbers ## to be used ## Return the mean of the pollutant across all monitors list ## in the 'id' vector (ignoring NA values) } 

这个函数的输出示例如下所示:

 source("pollutantmean.R") pollutantmean("specdata", "sulfate", 1:10) ## [1] 4.064 pollutantmean("specdata", "nitrate", 70:72) ## [1] 1.706 pollutantmean("specdata", "nitrate", 23) ## [1] 1.281 

我可以一口气读完整件事情:

 path = "C:/Users/Sean/Documents/R Projects/Data/specdata" fileList = list.files(path=path,pattern="\\.csv$",full.names=T) all.files.data = lapply(fileList,read.csv,header=TRUE) DATA = do.call("rbind",all.files.data) 

我的问题是:

  1. 用户inputid或者primefaces或者在一个范围内,例如,假设用户input1,但是文件名是001.csv,或者如果用户input一个范围1:10,那么文件名是001.csv … 010.csv
  2. 列是由用户,即“硫酸盐”或“硝酸盐”,他/她有兴趣得到的意思…这些列中有很多缺失的值(我需要在计算平均值之前从列中省略)。

所有文件的全部数据如下所示:

 summary(DATA) Date sulfate nitrate ID 2004-01-01: 250 Min. : 0.0 Min. : 0.0 Min. : 1.0 2004-01-02: 250 1st Qu.: 1.3 1st Qu.: 0.4 1st Qu.: 79.0 2004-01-03: 250 Median : 2.4 Median : 0.8 Median :168.0 2004-01-04: 250 Mean : 3.2 Mean : 1.7 Mean :164.5 2004-01-05: 250 3rd Qu.: 4.0 3rd Qu.: 2.0 3rd Qu.:247.0 2004-01-06: 250 Max. :35.9 Max. :53.9 Max. :332.0 (Other) :770587 NA's :653304 NA's :657738 

任何想法如何制定这将高度赞赏…

干杯

所以,你可以像这样模拟你的情况;

 # Simulate some data: # Create 332 data frames set.seed(1) df.list<-replicate(332,data.frame(sulfate=rnorm(100),nitrate=rnorm(100)),simplify=FALSE) # Generate names like 001.csv and 010.csv file.names<-paste0('specdata/',sprintf('%03d',1:332),'.csv') # Write them to disk invisible(mapply(write.csv,df.list,file.names)) 

这里是一个读取这些文件的函数:

 pollutantmean <- function(directory, pollutant, id = 1:332) { file.names <- list.files(directory) file.numbers <- as.numeric(sub('\\.csv$','', file.names)) selected.files <- na.omit(file.names[match(id, file.numbers)]) selected.dfs <- lapply(file.path(directory,selected.files), read.csv) mean(c(sapply(selected.dfs, function(x) x[ ,pollutant])), na.rm=TRUE) } pollutantmean('specdata','nitrate',c(1:100,141)) # [1] -0.005450574 
 User enters id either atomic or in a range eg 

假设用户input1,但是文件名是001.csv,或者如果用户input一个范围1:10,那么文件名是001.csv … 010.csv

您可以使用正则expression式和gsub函数从文件名中删除前导零,然后创build一个字典(在r中是一个命名向量),将修改的/ gsub'd文件名转换为实际的文件名。 例如:如果你的文件名是在一个字符向量, fnames

 fnames = c("001.csv","002.csv") names(fnames) <- gsub(pattern="^[0]*", replacement="", x=fnames) 

有了这个,vectorfnames被转换成一个字典,让你用fnames["1.csv"]调用名为001.csv的文件。 您也可以使用gsub()来删除文件名的.csv部分。

列是由用户,即“硫酸盐”或“硝酸盐”,他/她有兴趣得到的意思…这些列中有很多缺失的值(我需要在计算平均值之前从列中省略)。

许多R函数都可以忽略指示缺less值的特殊字符。 尝试在R命令提示符下inputhelp(mean)来查找有关此function的信息。

这就是我解决这个问题的方法:

 pollutantmean <- function(directory, pollutant, id = 1:332) { #set the path path = directory #get the file List in that directory fileList = list.files(path) #extract the file names and store as numeric for comparison file.names = as.numeric(sub("\\.csv$","",fileList)) #select files to be imported based on the user input or default selected.files = fileList[match(id,file.names)] #import data Data = lapply(file.path(path,selected.files),read.csv) #convert into data frame Data = do.call(rbind.data.frame,Data) #calculate mean mean(Data[,pollutant],na.rm=TRUE) } 

最后一个问题是我的函数应该调用“specdata”(所有csv所在的目录名称)作为目录,在r中是否有目录types对象?

假设我把这个函数称为:

 pollutantmean(specdata, "niterate", 1:10) 

它应该得到我的工作目录specdata目录的path…我怎么能做到这一点?

这是一个有点通用的函数,用于计算文件列表上特定列的平均值。 不知道应该如何设置id ,但现在它作为一个索引向量(即id = 1:3计算文件列表中前三个文件的平均值)。

 multifile.means <- function(directory = getwd(), pollutant, id = NULL) { d <- match.arg(directory, list.files()) cn <- match.arg(pollutant, c('sulfate', 'nitrate')) ## get a vector of complete file paths in the given 'directory' p <- dir(d, full.names = TRUE) ## subset 'p' based on 'id' values if(!is.null(id)){ id <- id[!id > length(p)] p <- p[id] } ## read, store, and name the relevant columns cl <- sapply(p, function(x){ read.csv(x)[,cn] }, USE.NAMES = FALSE) colnames(cl) <- basename(p) ## return a named list of some results list(values = cl, mean = mean(cl, na.rm = TRUE), colMeans = colMeans(cl, na.rm = TRUE)) } 

拿它来试驾:

 > multifile.means('testDir', 'sulfate') # $values # 001.csv 057.csv 146.csv 213.csv # [1,] 5 10 NA 9 # [2,] 1 1 10 3 # [3,] 10 4 10 2 # [4,] 3 10 9 NA # [5,] 4 1 5 5 # $mean # [1] 5.666667 # $colMeans # 001.csv 057.csv 146.csv 213.csv # 4.60 5.20 8.50 4.75 

这是一个解决scheme,即使你的祖母也能理解:

 pollutantmean <- function(directory, pollutant, id = 1:332) { # Break this function up into a series of smaller functions # that do exactly what you expect them to. Your friends # will love you for it. csvFiles = getFilesById(id, directory) dataFrames = readMultipleCsvFiles(csvFiles) dataFrame = bindMultipleDataFrames(dataFrames) getColumnMean(dataFrame, column = pollutant) } getFilesById <- function(id, directory = getwd()) { allFiles = list.files(directory) file.path(directory, allFiles[id]) } readMultipleCsvFiles <- function(csvFiles) { lapply(csvFiles, read.csv) } bindMultipleDataFrames <- function(dataFrames) { Reduce(function(x, y) rbind(x, y), dataFrames) } getColumnMean <- function(dataFrame, column, ignoreNA = TRUE) { mean(dataFrame[ , column], na.rm = ignoreNA) } 

选定的答案看起来不错,但是这里有一个select。 这个答案适用于JHU课程的基础知识。

 pollutantmean <- function(directory, pollutant, id = 1:332) { csvfiles <- dir(directory, "*\\.csv$", full.names = TRUE) data <- lapply(csvfiles[id], read.csv) numDataPoints <- 0L total <- 0L for (filedata in data) { d <- filedata[[pollutant]] # relevant column data d <- d[complete.cases(d)] # remove NA values numDataPoints <- numDataPoints + length(d) total <- total + sum(d) } total / numDataPoints } 

我也在阅读课程,并提出了以下解决scheme:

 pollutantmean <- function(directory="d:/dev/r/documents/specdata", pollutant, id) { myfilename = paste(directory,"/",formatC(id, width=3, flag="0"),".csv", sep="") master = lapply(myfilename, read.table, header=TRUE, sep=",") masterfile = do.call("rbind", master) head(masterfile[[2]], 100) if (pollutant == "sulfate") { #result=lapply(masterfile[[2]], mean, na.rm=TRUE) result=mean(masterfile[[2]], na.rm=TRUE) } if (pollutant == "nitrate") { result=mean(masterfile[[3]], na.rm=TRUE) } result } 

我花了几个小时来解决这个问题,但这里是我的(较短的)版本

 pollutmean<- function(dir, pollutant, id=1:332) { dir<- list.files(dir, full.names = T) #list files dat<- data.frame() #make empty df for (i in id) { dat <- rbind(dat, read.csv(dir[i])) #rbind all files } mean(dat[,pollutant], na.rm = TRUE) #calculate mean of given column } pollutmean("assign/specdata", "sulfate", id=1:60)