diff --git a/DESCRIPTION b/DESCRIPTION index 200bf1f..2e86ed7 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -45,7 +45,11 @@ Depends: truncnorm, OpenRepGrid Suggests: - shiny + shiny, + testthat (>= 2.1.0), + utils URL: https://github.com/unhcr/koboloadeR BugReports: https://github.com/unhcr/koboloadeR/issues RoxygenNote: 6.1.1 +VignetteBuilder: utils + diff --git a/NAMESPACE b/NAMESPACE index b81d846..7ccc7f0 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -43,6 +43,7 @@ export(kobo_histo) export(kobo_histo_print) export(kobo_indicator) export(kobo_label) +export(kobo_left_align) export(kobo_load_data) export(kobo_load_packages) export(kobo_map_cat) @@ -65,6 +66,10 @@ export(kobo_time_parser_UTC) export(kobo_to_xlsform) export(kobo_trend) export(kobo_trend_report) +export(kobo_unhcr_style_bar) +export(kobo_unhcr_style_histo) +export(kobo_unhcr_style_map) +export(kobo_unhcr_style_scatter) export(kobo_weight) export(ltbl) export(multresponse) diff --git a/R/kobo_anonymise.R b/R/kobo_anonymise.R index 3703545..220daa5 100644 --- a/R/kobo_anonymise.R +++ b/R/kobo_anonymise.R @@ -35,8 +35,6 @@ #' #' @author Edouard Legoupil #' -#' @examples -#' kobo_anonymise() #' #' @export kobo_anonymise #' diff --git a/R/kobo_create_indicators.R b/R/kobo_create_indicators.R index 0bd600b..cf3ec49 100644 --- a/R/kobo_create_indicators.R +++ b/R/kobo_create_indicators.R @@ -6,7 +6,7 @@ #' #' @param form The full filename of the form to be accessed (xls or xlsx file). #' It is assumed that the form is stored in the data folder. -#' +#' #' #' @return No return, all results will be saved inside new CSV files #' @@ -24,43 +24,65 @@ #' kobo_create_indicators <- function(form = "form.xls") { + + mainDir <- kobo_getMainDirectory() + form_tmp <- paste(mainDir, "data", form, sep = "/", collapse = "/") + tryCatch({ +<<<<<<< HEAD + +======= mainDir <- kobo_getMainDirectory() form_tmp <- paste(mainDir, "data", form, sep = "/", collapse = "/") - + ## Load all required packages kobo_load_packages() library(koboloadeR) - + ## load all required data files ######################################### cat("\n\nload all required data files..\n") dataBeginRepeat <- kobo_get_begin_repeat() dataBeginRepeat <- dataBeginRepeat$names for (dbr in dataBeginRepeat) { - dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""),stringsAsFactors = F) + dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""),stringsAsFactors = F) assign(dbr, dataFrame) } - + +>>>>>>> fffa73afd5387d145dea1058ded0957512987da9 #### Load and test i indicators ############################################################################# #library(readxl) tried <- try(read_excel(form_tmp, sheet = "indicator"), silent = TRUE) if (inherits(tried, "try-error")) { - writeLines("There was an error: You have not defined indicators within your xlsform file. \n") - + writeLines("Note that you have not defined (or defined correctly) indicators within your xlsform file. \n") + } else { - + rm(tried) + ## Load all required packages + kobo_load_packages() + # library(koboloadeR) + + ## load all required data files ######################################### + cat("\n\nload all required data files..\n") + dataBeginRepeat <- kobo_get_begin_repeat() + dataBeginRepeat <- dataBeginRepeat$names + for (dbr in dataBeginRepeat) { + dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"-edited.csv",sep = ""),stringsAsFactors = F) + assign(dbr, dataFrame) + } + + indicator <- read_excel(form_tmp, sheet = "indicator") if(nrow(indicator)==0){ - writeLines("There was an error: You have not defined indicators within your xlsform file. \n") - + writeLines("Note that you have not defined (or defined correctly) indicators within your xlsform file. \n") + } else { ## Load data & dico ############################################################################# #form <- "form.xls" ## Run this only after data cleaning dico <- read.csv(paste0(mainDir,"/data/dico_",form,".csv"), encoding = "UTF-8", na.strings = "") - + ## Create the dicotemp ############################################################################# #names(dico) dicotemp <- data.frame(c("trigger")) @@ -68,22 +90,26 @@ kobo_create_indicators <- function(form = "form.xls") { #dicotemp$type <- "trigger" dicotemp$name <- "trigger" dicotemp$fullname <- "trigger" - dicotemp$label <- "trigger" + # dicotemp$label <- "trigger" dicotemp$labelReport <- "trigger" + dicotemp$hintReport <- "trigger" dicotemp$chapter <- "trigger" dicotemp$disaggregation <- "trigger" dicotemp$correlate <- "trigger" dicotemp$anonymise <- "trigger" - - dicotemp$structuralequation <- "trigger" + + dicotemp$structuralequation.risk <- "trigger" + dicotemp$structuralequation.coping <- "trigger" + dicotemp$structuralequation.resilience <- "trigger" + dicotemp$clean <- "trigger" dicotemp$cluster <- "trigger" dicotemp$predict <- "trigger" dicotemp$variable <- "trigger" dicotemp$mappoint <- "trigger" dicotemp$mappoly <- "trigger" - - + + dicotemp$listname <- "trigger" dicotemp$qrepeat <- "trigger" dicotemp$qrepeatlabel <- "trigger" @@ -97,105 +123,112 @@ kobo_create_indicators <- function(form = "form.xls") { dicotemp$recategorise <- "trigger" dicotemp$formpart <- "trigger" dicotemp$indic <- "feature" - - ####Load data analysis plan############################################################################# - #library(readxl) - indicator <- read_excel(form_tmp, sheet = "indicator") - + + ## Need to check that all column are presents... - - + + ## Load indicator info ############################################################################# - + for (i in 1:nrow(indicator)) - + { + # i <- 1 indicator.type <- as.character(indicator[ i, c("type")]) indicator.fullname <- as.character(indicator[ i, c("fullname")]) - indicator.label <- as.character(indicator[ i, c("label")]) - indicator.report <- as.character(indicator[ i, c("label")]) + # indicator.label <- as.character(indicator[ i, c("label")]) + indicator.labelReport <- as.character(indicator[ i, c("labelReport")]) + indicator.hintReport <- as.character(indicator[ i, c("hintReport")]) indicator.chapter <- as.character(indicator[ i, c("chapter")]) indicator.disaggregation <- as.character(indicator[ i, c("disaggregation")]) indicator.correlate <- as.character(indicator[ i, c("correlate")]) - indicator.sensitive <- as.character(indicator[ i, c("sensitive")]) indicator.anonymise <- as.character(indicator[ i, c("anonymise")]) indicator.frame <- as.character(indicator[ i, c("frame")]) indicator.listname <- as.character(indicator[ i, c("listname")]) indicator.calculation <- as.character(indicator[ i, c("calculation")]) - - - indicator.structuralequation <- as.character(indicator[ i, c("structuralequation")]) + + + indicator.structuralequation.risk <- as.character(indicator[ i, c("structuralequation.risk")]) + indicator.structuralequation.coping <- as.character(indicator[ i, c("structuralequation.coping")]) + indicator.structuralequation.resilience <- as.character(indicator[ i, c("structuralequation.resilience")]) indicator.cluster <- as.character(indicator[ i, c("cluster")]) indicator.predict <- as.character(indicator[ i, c("predict")]) indicator.variable <- as.character(indicator[ i, c("variable")]) indicator.mappoint <- as.character(indicator[ i, c("mappoint")]) indicator.mappoly <- as.character(indicator[ i, c("mappoly")]) - - - cat(paste0(i, "- Load indicator: ", indicator.label," of type: ",indicator.type,"\n")) - + + + cat(paste0(i, "- Load indicator: ", indicator.labelReport," of type: ",indicator.type,"\n")) + ## Build and run the formula to insert the indicator in the right frame ########################### indic.formula <- paste0(indicator.frame,"$",indicator.fullname," <- ",indicator.calculation ) if (file.exists(paste0(mainDir,"/code/temp.R") )) file.remove(paste0(mainDir,"/code/temp.R")) + + + cat(paste('### Script to generate indicator: ',indicator.labelReport,sep = ""), file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) cat(paste('form <- "',form,'"',sep = ""), file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) cat("mainDir <- kobo_getMainDirectory()", file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) cat('form_tmp <- paste(mainDir, "data", form, sep = "/", collapse = "/")', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) cat('dataBeginRepeat <- kobo_get_begin_repeat()', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) cat('dataBeginRepeat <- dataBeginRepeat$names', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) - cat('MainDataFrame_edited <- read.csv(paste(mainDir,"/data/MainDataFrame_edited.csv",sep = ""), encoding = "UTF-8", na.strings = "NA")', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) - - cat(' - for (dbr in dataBeginRepeat) { - dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""),stringsAsFactors = F) + + cat('MainDataFrame <- read.csv(paste(mainDir,"/data/MainDataFrame-edited.csv",sep = ""), encoding = "UTF-8", na.strings = "NA")', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) + + cat('for (dbr in dataBeginRepeat) { + dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"-edited.csv",sep = ""),stringsAsFactors = F) + assign(dbr, dataFrame) - } - ', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) - + }', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) + cat(indic.formula, file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) cat("####", file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) - + ## do a check on indicator variable type indicator.type2 <- indicator.type ifelse(indicator.type == "select_one", indicator.type2 <- "character", indicator.type2 <- indicator.type) - - + + cat(paste0(indicator.frame,"$",indicator.fullname," <- as.",indicator.type2,"(",indicator.frame,"$",indicator.fullname,")"), file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) cat(paste0("str(",indicator.frame,"$",indicator.fullname,")"), file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) cat(paste0("summary(",indicator.frame,"$",indicator.fullname,")"), file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) - - if(indicator.frame == "MainDataFrame_edited"){ - cat('write.csv(MainDataFrame_edited, paste(mainDir,"/data/MainDataFrame_edited.csv",sep = ""), row.names = FALSE, na = "")', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) - }else{ - cat(paste('dbr<-"',indicator.frame,'"',sep = "")) - cat('write.csv(eval(as.name(dbr)),paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""), row.names = FALSE, na = "")', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) - } - + + if (indicator.frame == "MainDataFrame") { + cat('write.csv(MainDataFrame, paste(mainDir,"/data/MainDataFrame-edited.csv",sep = ""), row.names = FALSE, na = "")', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) + }else{ + cat(paste('dbr<-"',indicator.frame,'"',sep = "")) + cat('write.csv(eval(as.name(dbr)),paste(mainDir,"/data/",dbr,"-edited.csv",sep = ""), row.names = FALSE, na = "")', file = paste0(mainDir,"/code/temp.R") , sep = "\n", append = TRUE) + } + + source(paste0(mainDir,"/code/temp.R")) - cat(paste0(i, "- Executed indicator: ", indicator.label,"\n")) + cat(paste0(i, "- Executed indicator: ", indicator.labelReport,"\n")) if (file.exists(paste0(mainDir,"/code/temp.R"))) file.remove(paste0(mainDir,"/code/temp.R")) - + ## Insert the indicator in a temp dico frame to be appended to the full dico ###################### - + dicotemp1 <- data.frame(c("trigger")) names(dicotemp1)[1] <- "type" dicotemp1$type <- indicator.type dicotemp1$name <- indicator.fullname dicotemp1$fullname <- indicator.fullname - dicotemp1$label <- indicator.label - dicotemp1$labelReport <- indicator.report + #dicotemp1$label <- indicator.label + dicotemp1$labelReport <- indicator.labelReport + dicotemp1$hintReport <- indicator.hintReport dicotemp1$chapter <- indicator.chapter dicotemp1$disaggregation <- indicator.disaggregation dicotemp1$correlate <- indicator.correlate dicotemp1$anonymise <- indicator.anonymise - - dicotemp1$structuralequation <- indicator.structuralequation + + dicotemp1$structuralequation.risk <- indicator.structuralequation.risk + dicotemp1$structuralequation.coping <- indicator.structuralequation.coping + dicotemp1$structuralequation.resilience <- indicator.structuralequation.resilience dicotemp1$clean <- " " dicotemp1$cluster <- indicator.cluster dicotemp1$predict <- indicator.predict dicotemp1$variable <- indicator.variable dicotemp1$mappoint <- indicator.mappoint dicotemp1$mappoly <- indicator.mappoly - + dicotemp1$listname <- indicator.listname dicotemp1$qrepeat <- " " dicotemp1$qrepeatlabel <- indicator.frame @@ -208,167 +241,187 @@ kobo_create_indicators <- function(form = "form.xls") { dicotemp1$recategorise <- " " dicotemp1$formpart <- " " dicotemp1$indic <- "feature" - + dicotemp <- rbind(dicotemp,dicotemp1) - + } ## Append indicators in the dico ############################################################################# - + ## removing first line dicotemp <- dicotemp[ 2:nrow(dicotemp), ] - + ### mergin choices from the newly created indicators ################################################################# - + cat("\n\n\n It's assumed that the modalities for newly calculated categoric indicators are in the same xlsform - choices worksheet \n\n\n\n") choices <- read_excel(form_tmp, sheet = "choices") - + #rm(choices) - names(choices)[names(choices) == "label::English"] <- "label" - names(choices)[names(choices) == "label::english"] <- "label" + names(choices)[names(choices) == "labelReport"] <- "label" + #names(choices)[names(choices) == "label::english"] <- "label" names(choices)[names(choices) == "list name"] <- "listname" names(choices)[names(choices) == "list_name"] <- "listname" - + ## Remove trailing space choices$listname <- trim(choices$listname) choices$label <- trim(choices$label) - + if ("labelReport" %in% colnames(choices)) { - cat("12 - Good: You have a column `labelReport` in your `choices` worksheet.\n"); + cat(" Good: You have a column `labelReport` in your `choices` worksheet.\n"); } else - {cat("12 - No column `labelReport` in your `choices` worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `labelReport` in your `choices` worksheet. Creating a dummy one for the moment...\n"); choices[,"labelReport"] <- substr(choices[,"label"],1,80)} - + if ("order" %in% colnames(choices)) { cat(" Good: You have a column `order` in your `choices` worksheet.\n"); } else {cat(" No column `order` in your `choices` worksheet. Creating a dummy one for the moment...\n"); choices$order <- ""} - + if ("weight" %in% colnames(choices)) { cat(" Good: You have a column `weight` in your `choices` worksheet.\n"); } else - {cat("13 - No column `weight` in your `choices` worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `weight` in your `choices` worksheet. Creating a dummy one for the moment...\n"); choices$weight <- ""} - + if ("recategorise" %in% colnames(choices)) { cat(" Good: You have a column `recategorise` in your `choices` worksheet.\n"); } else {cat(" No column `recategorise` in your `choices` worksheet. Creating a dummy one for the moment...\n"); choices$recategorise <- ""} - + if ("score" %in% colnames(choices)) { cat(" Good: You have a column `score` in your `choices` worksheet.\n"); } else {cat(" No column `score` in your `choices` worksheet. Creating a dummy one for the moment...\n"); choices$score <- ""} - + choices <- choices[,c("listname", "name", "label", "order", "weight","score","recategorise")] names(choices)[names(choices) == "label"] <- "labelchoice" #rm(choices) - - dicotemp.choice <- dicotemp[ !(is.na(dicotemp$listname)), c( "type", "name", "fullname", "label", "labelReport", + + dicotemp.choice <- dicotemp[ !(is.na(dicotemp$listname)), c( "type", "name", "fullname", "label", "labelReport","hintReport", "chapter", "disaggregation","correlate", "anonymise", - "structuralequation", "clean", "cluster", "predict", + "structuralequation.risk","structuralequation.coping","structuralequation.resilience", + "clean", "cluster", "predict", "variable", "mappoint", "mappoly", "listname", "qrepeat", "qrepeatlabel","qlevel","qgroup" )] - + choices2 <- join(x = dicotemp.choice, y = choices, by = "listname", type = "left") - + choices2$type <- with(choices2, ifelse(grepl("select_one", ignore.case = TRUE, fixed = FALSE, useBytes = FALSE, choices2$type), paste0("select_one_d"),choices2$type)) - + choices2$type <- with(choices2, ifelse(grepl("select_multiple_d", ignore.case = TRUE, fixed = FALSE, useBytes = FALSE, choices2$type), paste0("select_multiple"),choices2$type)) - + names(choices2)[2] <- "nameq" names(choices2)[3] <- "nameqfull" names(choices2)[4] <- "labelq" choices2$labelfull <- paste0(choices2$labelq, sep = ": ", choices2$labelchoice) choices2$namefull <- paste0(choices2$nameqfull, sep = ".", choices2$name) - - - + + + #### Now Row bind questions & choices######################################################################################################## - choices3 <- choices2[ ,c("type", "name", "namefull", "labelfull", "labelReport", + choices3 <- choices2[ ,c("type", "name", "namefull", "labelfull", "labelReport","hintReport", "chapter", "disaggregation","correlate", "anonymise", - "structuralequation", "clean", "cluster", "predict", + "structuralequation.risk","structuralequation.coping","structuralequation.resilience", + "clean", "cluster", "predict", "variable", "mappoint", "mappoly", "listname", "qrepeat", "qrepeatlabel","qlevel","qgroup", "labelchoice", "order", "weight","score", "recategorise")] - - + + names(choices3)[names(choices3) == "namefull"] <- "fullname" names(choices3)[names(choices3) == "labelfull"] <- "label" - - - dicotemp <- dicotemp[,c( "type", "name", "fullname", "label", "labelReport", + + + dicotemp <- dicotemp[,c( "type", "name", "fullname", "label", "labelReport","hintReport", "chapter", "disaggregation","correlate", "anonymise", - "structuralequation", "clean", "cluster", "predict", + "structuralequation.risk","structuralequation.coping","structuralequation.resilience", + "clean", "cluster", "predict", "variable", "mappoint", "mappoly", "listname", "qrepeat", "qrepeatlabel","qlevel","qgroup", "labelchoice", "order", "weight","score", "recategorise")] - + ### Check -- normally there should not be duplicate - - + + dicotemp$formpart <- "questions" choices3$formpart <- "answers" - + dicotemp <- rbind(dicotemp,choices3) - - + + dicotemp$indic <- "feature" dico$indic <- "data" - - dico$structuralequation <- NA - + + + dicotemp$relevant <- "" + dicotemp$required <- "" + dicotemp$constraint <- "" + dicotemp$repeat_count <- "" + + #names(dico) #names(dicotemp) - dico <- dico[ , c( "type", "name", "fullname", "label", "labelReport", + dico <- dico[ , c( "type", "name", "fullname", "label", "labelReport","hintReport", "chapter", "disaggregation","correlate", "anonymise", - "structuralequation", "clean", "cluster", "predict", - "variable", "mappoint", "mappoly", "listname", - "qrepeat", "qrepeatlabel","qlevel","qgroup", + "structuralequation.risk","structuralequation.coping","structuralequation.resilience", + "anonymise", "clean", "cluster", "predict", "variable", "mappoint", "mappoly", + + "relevant", "required", "constraint", "repeat_count", + + "listname","qrepeat", "qrepeatlabel","qlevel","qgroup", "labelchoice", "order", "weight","score", "recategorise", "formpart", "indic" )] - dicotemp <- dicotemp[ , c( "type", "name", "fullname", "label", "labelReport", - "chapter", "disaggregation","correlate", "anonymise", - "structuralequation", "clean", "cluster", "predict", - "variable", "mappoint", "mappoly", "listname", - "qrepeat", "qrepeatlabel","qlevel","qgroup", - "labelchoice", "order", "weight","score", - "recategorise", "formpart", "indic" )] - - + + + dicotemp <- dicotemp[ , c("type", "name", "fullname", "label", "labelReport","hintReport", + "chapter", "disaggregation","correlate", "anonymise", + "structuralequation.risk","structuralequation.coping","structuralequation.resilience", + "anonymise", "clean", "cluster", "predict", "variable", "mappoint", "mappoly", + + "relevant", "required", "constraint", "repeat_count", + + "listname","qrepeat", "qrepeatlabel","qlevel","qgroup", + "labelchoice", "order", "weight","score", + "recategorise", "formpart", "indic" )] + + dico <- rbind(dico,dicotemp) - + rm(dicotemp,dicotemp1, choices, choices2, choices3, dicotemp.choice) - - MainDataFrame_edited <- read.csv(paste(mainDir,"/data/MainDataFrame_edited.csv",sep = ""), encoding = "UTF-8", na.strings = "NA") + + + MainDataFrame <- read.csv(paste(mainDir,"/data/MainDataFrame-edited.csv",sep = ""), encoding = "UTF-8", na.strings = "NA") ## label Variables cat("\n\n quick check on labeling\n") - MainDataFrame_edited <- kobo_label(MainDataFrame_edited , dico) + MainDataFrame <- kobo_label(MainDataFrame , dico) for (dbr in dataBeginRepeat) { - dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""),stringsAsFactors = F) + dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"-edited.csv",sep = ""),stringsAsFactors = F) + dataFrame <- kobo_label(dataFrame, dico) write.csv(dataFrame,paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""), row.names = FALSE, na = "") } cat("\n\nWrite dico\n") write.csv(dico, paste0(mainDir,"/data/dico_",form,".csv"), row.names = FALSE, na = "") - write.csv(MainDataFrame_edited, paste(mainDir,"/data/MainDataFrame_edited.csv",sep = ""), row.names = FALSE, na = "") - + + write.csv(MainDataFrame, paste(mainDir,"/data/MainDataFrame-edited.csv",sep = ""), row.names = FALSE, na = "") + + } } }, error = function(err) { - print("kobo_create_indicators_ERROR") + print("There as an error in the indicator creation step!!! \n\n") return(structure(err, class = "try-error")) }) -} \ No newline at end of file +} diff --git a/R/kobo_crunching_report.R b/R/kobo_crunching_report.R index 049d932..1ecdc26 100644 --- a/R/kobo_crunching_report.R +++ b/R/kobo_crunching_report.R @@ -22,7 +22,7 @@ #' @export kobo_crunching_report #' -kobo_crunching_report <- function(form = "form.xls", app="console") { +kobo_crunching_report <- function(form = "form.xls", app = "console") { tryCatch({ if (app == "shiny") { progress <- shiny::Progress$new() @@ -43,11 +43,13 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { configInfo <- configInfo[!is.na(configInfo$name),] mainDir <- kobo_getMainDirectory() form_tmp <- paste(mainDir, "data", form, sep = "/", collapse = "/") - library(koboloadeR) + #library(koboloadeR) ### Load the data cat("\n\n Loading data. It is assumed that the cleaning, weighting & re-encoding has been done previously \n") - MainDataFrame_edited <- read.csv(paste(mainDir,"/data/MainDataFrame.csv",sep = ""), encoding = "UTF-8", na.strings = "") + + MainDataFrame <- read.csv(paste(mainDir,"/data/MainDataFrame-encoded.csv",sep = ""), encoding = "UTF-8", na.strings = "") + ###Form########################################## ## Load form @@ -66,13 +68,17 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { progress$set(message = "Labelling variables in the Main Data File in progress...") updateProgress() } - MainDataFrame_edited <- kobo_label(MainDataFrame_edited , dico) + + MainDataFrame <- kobo_label(MainDataFrame , dico) + cat("\n\nload all required data files..\n") dataBeginRepeat <- kobo_get_begin_repeat() dataBeginRepeat <- dataBeginRepeat$names for (dbr in dataBeginRepeat) { - dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""),stringsAsFactors = F) + + dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"-encoded.csv",sep = ""),stringsAsFactors = F) + assign(dbr, kobo_label(dataFrame, dico)) if (app == "shiny") { progress$set(message = paste("Labelling variables in",dbr,"File in progress...")) @@ -130,6 +136,7 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { for (i in 1:nrow(chapters) ) { + # i <-1 chaptersname <- as.character(chapters[ i , 1]) if (app == "shiny") { progress$set(message = paste(i, " - Render chapter for ",as.character(chapters[ i , 1]))) @@ -145,7 +152,7 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { ## TO DO : put in configuration wethere report should be portrait or landscape cat("---", file = chapter.name , sep = "\n", append = TRUE) cat(paste("title: \"Data Crunching Report: ",chaptersname , "- Draft not for distribution. \"", sep = ""), file = chapter.name , sep = "\n", append = TRUE) - cat("author: \"Generated with [Koboloader](https://github.com/unhcr/koboloadeR) \"", file = chapter.name , sep = "\n", append = TRUE) + cat("author: \"Generated with [Koboloader](https://unhcr.github.io/koboloadeR/docs) \"", file = chapter.name , sep = "\n", append = TRUE) cat("date: \" `r format(Sys.Date(), '%d %B %Y')`\"", file = chapter.name , sep = "\n", append = TRUE) cat("always_allow_html: yes", file = chapter.name , sep = "\n", append = TRUE) cat("output:",file = chapter.name , sep = "\n", append = TRUE) @@ -166,6 +173,7 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat("mainDirroot <- substring(mainDir, 0 , nchar(mainDir) - 5)", file = chapter.name , sep = "\n", append = TRUE) + cat("## Load all required packages", file = chapter.name , sep = "\n", append = TRUE) cat("library(tidyverse)", file = chapter.name , sep = "\n", append = TRUE) cat("library(ggthemes)", file = chapter.name , sep = "\n", append = TRUE) cat("library(plyr)", file = chapter.name , sep = "\n", append = TRUE) @@ -179,28 +187,33 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat("library(survey)", file = chapter.name , sep = "\n", append = TRUE) cat("library(knitr)", file = chapter.name , sep = "\n", append = TRUE) cat("library(rmarkdown)", file = chapter.name , sep = "\n", append = TRUE) - - #("source(paste0(mainDirroot,\"/code/0-theme.R\"))", file = chapter.name , sep = "\n", append = TRUE) + cat("library(ggpubr)", file = chapter.name , sep = "\n", append = TRUE) + cat("library(grid)", file = chapter.name , sep = "\n", append = TRUE) cat("library(koboloadeR)", file = chapter.name , sep = "\n", append = TRUE) - cat("## Load all required packages", file = chapter.name , sep = "\n", append = TRUE) - #cat("kobo_load_data()", file = chapter.name , sep = "\n", append = TRUE) - # cat("source(paste0(mainDirroot,\"/code/0-packages.R\"))", file = chapter.name , sep = "\n", append = TRUE) + cat("options(scipen = 999) # turn-off scientific notation like 1e+48", file = chapter.name , sep = "\n", append = TRUE) + cat("## Provide below the name of the form in xsl form - format should be xls not xlsx", file = chapter.name , sep = "\n", append = TRUE) cat(paste0("form <- \"",form,"\""), file = chapter.name , sep = "\n", append = TRUE) cat("dico <- read.csv(paste0(mainDirroot,\"/data/dico_\",form,\".csv\"), encoding = \"UTF-8\", na.strings = \"\")", file = chapter.name , sep = "\n", append = TRUE) ## TO DO: Use config file to load the different frame - cat("MainDataFrame_edited <- read.csv(paste0(mainDirroot,\"/data/MainDataFrame_edited.csv\"), encoding = \"UTF-8\", na.strings = \"\")", file = chapter.name , sep = "\n", append = TRUE) + + + cat("MainDataFrame <- read.csv(paste0(mainDirroot,\"/data/MainDataFrame-encoded.csv\"), encoding = \"UTF-8\", na.strings = \"\")", file = chapter.name , sep = "\n", append = TRUE) + for (dbr in dataBeginRepeat) { - cat(paste(dbr, "<- read.csv(paste0(mainDirroot,\"/data/",dbr,"_edited.csv\"), encoding = \"UTF-8\", na.strings = \"\")", sep = ""), file = chapter.name , sep = "\n", append = TRUE) + cat(paste(dbr, " <- read.csv(paste0(mainDirroot,\"/data/",dbr,"-encoded.csv\"), encoding = \"UTF-8\", na.strings = \"\")", sep = ""), file = chapter.name , sep = "\n", append = TRUE) + } cat("\n", file = chapter.name , sep = "\n", append = TRUE) cat("## label Variables", file = chapter.name , sep = "\n", append = TRUE) - cat("MainDataFrame_edited <- kobo_label(MainDataFrame_edited , dico)", file = chapter.name , sep = "\n", append = TRUE) + + cat("MainDataFrame <- kobo_label(MainDataFrame , dico)", file = chapter.name , sep = "\n", append = TRUE) + for (dbr in dataBeginRepeat) { cat(paste(dbr, " <- kobo_label(",dbr ," , dico)", sep = ""), file = chapter.name , sep = "\n", append = TRUE) } @@ -227,28 +240,37 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat("\n", file = chapter.name , sep = "\n", append = TRUE) cat("## Create weighted survey object", file = chapter.name , sep = "\n", append = TRUE) + ## If no weight, the weighted object is unweigthted - if(configInfo[configInfo$name == "sample_type","value"] == "No sampling(type 1)"){ + if (configInfo[configInfo$name == "sample_type","value"] == "No sampling (type 1)") { ## If no weight, the weighted object is unweigthted - cat("MainDataFrame_edited.survey <- svydesign(ids = ~ 1 , data = MainDataFrame_edited )", file = chapter.name , sep = "\n", append = TRUE) + cat("MainDataFrame.survey <- svydesign(ids = ~ 1 , data = MainDataFrame )", file = chapter.name , sep = "\n", append = TRUE) + for (dbr in dataBeginRepeat) { cat(paste(dbr,".survey <- svydesign(ids = ~ 1 , data = ",dbr," )", sep = ""), file = chapter.name , sep = "\n", append = TRUE) } - }else if(configInfo[configInfo$name == "sample_type","value"] == "Cluster sample (type 2)"){ ## with clusters - cat(paste("MainDataFrame_edited.survey <- svydesign(ids = ~ ", configInfo[configInfo$name == "variable_name","value"],", data = MainDataFrame_edited, weights = ~ ", configInfo[configInfo$name == "weightsVariable","value"]," , fpc = ~ fpc )", sep = ""), file = chapter.name , sep = "\n", append = TRUE) + + }else if (configInfo[configInfo$name == "sample_type","value"] == "Cluster sample (type 2)") { + ## with clusters + cat(paste("MainDataFrame.survey <- svydesign(ids = ~ ", configInfo[configInfo$name == "variable_name","value"],", data = MainDataFrame, weights = ~ ", configInfo[configInfo$name == "weightsVariable","value"]," , fpc = ~ fpc )", sep = ""), file = chapter.name , sep = "\n", append = TRUE) + for (dbr in dataBeginRepeat) { cat(paste(dbr,".survey <- svydesign(ids = ~ ", configInfo[configInfo$name == "variable_name","value"],", data = ",dbr,", weights = ~ ", configInfo[configInfo$name == "weightsVariable","value"]," , fpc = ~ fpc )", sep = ""), file = chapter.name , sep = "\n", append = TRUE) } - }else if(configInfo[configInfo$name == "sample_type","value"] == "Stratified sample (type 3)"){ ## with strata - cat(paste("MainDataFrame_edited.survey <- svydesign(id=~1, strata= ~ ", configInfo[configInfo$name == "variable_name","value"]," ,check.strata = TRUE, data = MainDataFrame_edited, weights = ~ ", configInfo[configInfo$name == "weightsVariable","value"]," )", sep=""), file = chapter.name , sep = "\n", append = TRUE) + + }else if (configInfo[configInfo$name == "sample_type","value"] == "Stratified sample (type 3)") { + ## with strata + cat(paste("MainDataFrame.survey <- svydesign(id=~1, strata= ~ ", configInfo[configInfo$name == "variable_name","value"]," ,check.strata = TRUE, data = MainDataFrame, weights = ~ ", configInfo[configInfo$name == "weightsVariable","value"]," )", sep = ""), file = chapter.name , sep = "\n", append = TRUE) + for (dbr in dataBeginRepeat) { cat(paste(dbr,".survey <- svydesign(id=~1, strata= ~ ", configInfo[configInfo$name == "variable_name","value"]," ,check.strata = TRUE, data = ",dbr,", weights = ~ ", configInfo[configInfo$name == "weightsVariable","value"]," )", sep = ""), file = chapter.name , sep = "\n", append = TRUE) } } + ## with strata #cat("MainDataFrame_edited.survey <- svydesign(id=~1, strata= ~ RecordCategory ,check.strata = TRUE, data = MainDataFrame_edited, weights = ~ WeightingCoefficient )", file = chapter.name , sep = "\n", append = TRUE) @@ -262,6 +284,7 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { # cat("br1.survey <- svydesign(ids = ~ 1 , data = br1 )", file = chapter.name , sep = "\n", append = TRUE) # cat("br2.survey <- svydesign(ids = ~ 1 , data = br2 )", file = chapter.name , sep = "\n", append = TRUE) + cat(paste0("\n```\n", sep = '\n'), file = chapter.name, append = TRUE) @@ -295,7 +318,7 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { ## Getting chapter questions ####### #chapterquestions <- dico[which(dico$chapter== chaptersname ), c("chapter", "name", "label", "type", "qrepeatlabel", "fullname","listname") ] chapterquestions <- dico[which(dico$chapter == chaptersname & dico$type %in% c("select_one","integer","select_multiple_d", "text","date", "numeric")), - c("chapter", "name", "label", "labelReport", "type", "qrepeatlabel", "fullname","listname","variable") ] + c("chapter", "name", "label", "labelReport","hintReport", "type", "qrepeatlabel", "fullname","listname","variable") ] #levels(as.factor(as.character(dico[which(!(is.na(dico$chapter)) & dico$formpart=="questions"), c("type") ]))) ##Loop.questions#################################################################################################### if (app == "shiny") { @@ -315,6 +338,7 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { questions.type <- as.character(chapterquestions[ j , c("type")]) questions.frame <- as.character(chapterquestions[ j , c("qrepeatlabel")]) questions.label <- as.character(chapterquestions[ j , c("labelReport")]) + questions.hint <- as.character(chapterquestions[ j , c("hintReport")]) questions.listname <- as.character(chapterquestions[ j , c("listname")]) questions.ordinal <- as.character(chapterquestions[ j , c("variable")]) if (is.na(questions.ordinal) ) {questions.ordinal <- "not.defined"} else {questions.ordinal <- questions.ordinal } @@ -328,9 +352,10 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { ## Now create para based on question type------- + cat(paste(questions.hint,"\n\n",sep = ""),file = chapter.name ,sep = "\n", append = TRUE) - ###selectone################################################################################################### + ###select one################################################################################################### if (questions.type == "select_one" ) { cat(paste("Single choice question ","\n\n",sep = ""),file = chapter.name ,sep = "\n", append = TRUE) @@ -342,17 +367,9 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { figheight <- as.integer(nrow(frequ)) if (figheight == 0) { figheight <- 1} else {figheight <- figheight/1.2} - cat(paste("### Tabulation" ,sep = ""),file = chapter.name ,sep = "\n", append = TRUE) - ## Open chunk - cat(paste0("```{r ", questions.name, ".tab, echo=FALSE, warning=FALSE, cache=FALSE, tidy = TRUE, message=FALSE, comment = \"\", fig.height=",figheight,", size=\"small\"}\n"), file = chapter.name, append = TRUE) - cat(paste("### Tabulation" ,sep = ""),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("##Compute contengency table"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("frequ <- as.data.frame(table(",questions.variable,"))"),file = chapter.name ,sep = "\n", append = TRUE) - #cat(paste0("if (nrow(frequ)==0){ cat(\"No response for this question\") } else{"),file = chapter.name ,sep = "\n", append = TRUE) - ## Check that there are responses to be displayed #### if (nrow(frequ) %in% c("0","1") ) { - cat(paste0("cat(\"No responses recorded for this question...\")"),file = chapter.name , sep = "\n", append = TRUE) + cat(paste0("cat(\"No responses or only one modality recorded for this question...\")"),file = chapter.name , sep = "\n", append = TRUE) cat("No responses recorded for this question...\n") # names(frequ)[2] <- "ccheck" @@ -360,9 +377,19 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { # } else if (sum(try) == 0) { # cat(paste0("cat(\"No responses recorded for this question...\")"),file = chapter.name , sep = "\n", append = TRUE) # cat("No responses recorded for this question...\n") - } else{ - cat(paste0("## display table"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("## Reorder factor"),file = chapter.name ,sep = "\n", append = TRUE) + } else { + + cat(paste("### Tabulation" ,sep = ""),file = chapter.name ,sep = "\n", append = TRUE) + ## Open chunk + cat(paste0("```{r ", questions.name, ".tab, echo=FALSE, warning=FALSE, cache=FALSE, tidy = TRUE, message=FALSE, comment = \"\", fig.height=",figheight,", size=\"small\"}\n"), file = chapter.name, append = TRUE) + cat(paste("### Tabulation" ,sep = ""),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("##Compute contengency table"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("frequ <- as.data.frame(table(",questions.variable,"))"),file = chapter.name ,sep = "\n", append = TRUE) + #cat(paste0("if (nrow(frequ)==0){ cat(\"No response for this question\") } else{"),file = chapter.name ,sep = "\n", append = TRUE) + + + # cat(paste0("## display table"),file = chapter.name ,sep = "\n", append = TRUE) + # cat(paste0("## Reorder factor"),file = chapter.name ,sep = "\n", append = TRUE) ## Check variable type to order the factor #### @@ -378,11 +405,11 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat(paste0("names(frequ)[1] <- \"", questions.shortname,"\""),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("kable(frequ, caption=\"__Table__:", questions.label,"\")"),file = chapter.name ,sep = "\n", append = TRUE) + # cat(paste0("kable(frequ, caption=\"__Table__:", questions.label,"\")"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("## Frequency table with NA in order to get non response rate"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("frequ1 <- as.data.frame(prop.table(table(", questions.variable,", useNA=\"ifany\")))"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("frequ1 <- as.data.frame(prop.table(table(", questions.variable,", useNA = \"ifany\")))"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("frequ1 <- frequ1[!(is.na(frequ1$Var1)), ]"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("frequ1 <- frequ1[!(frequ1$Var1==\"NA\"), ]"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("frequ1 <- frequ1[!(frequ1$Var1 == \"NA\"), ]"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("percentreponse <- paste0(round(sum(frequ1$Freq)*100,digits = 1),\"%\")"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("## Frequency table without NA"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("frequ2 <- as.data.frame(prop.table(table(", questions.variable,",useNA = \"no\")))"),file = chapter.name ,sep = "\n", append = TRUE) @@ -410,18 +437,19 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat(paste0("\n"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("## and now the graph"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("ggplot(frequ3, aes(x=frequ3$Var1, y=frequ3$mean)) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("geom_bar(fill=\"#2a87c8\",colour=\"#2a87c8\", stat =\"identity\", width=.8) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("guides(fill=FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(frequ3, aes(x = frequ3$Var1, y = frequ3$mean)) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("geom_bar(fill = \"#2a87c8\", colour = \"#2a87c8\", stat = \"identity\", width=.8) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("guides(fill = FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_label_repel(aes(y = mean, label = freqper2), fill = \"#2a87c8\", color = 'white') +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ylab(\"Frequency\") +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("scale_y_continuous(labels=percent) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("scale_y_continuous(labels = percent) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("xlab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("coord_flip() +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"",questions.label,"\","),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("subtitle = paste0(\" Question response rate: \",percentreponse,\" .\")) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("theme(plot.title=element_text(face=\"bold\", size = 9 ),"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("plot.background = element_rect(fill = \"transparent\",colour = NA))"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("kobo_unhcr_style_bar()"),file = chapter.name ,sep = "\n", append = TRUE) + + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) } @@ -437,7 +465,7 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat("\n", file = chapter.name, append = TRUE) } else if (nrow(frequ) %in% c("0","1")) { # cat("No responses recorded for this question. No disaggregation...\n",file = chapter.name , sep = "\n", append = TRUE) - cat("No responses recorded for this question. No disaggregation...\n") + cat("No responses or only one modality recorded for this question. No disaggregation...\n") cat("\n", file = chapter.name, append = TRUE) } else { @@ -510,33 +538,33 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat(paste0("levels(",questions.frame,"$",disag.name,") <- list.ordinal"),file = chapter.name ,sep = "\n", append = TRUE) } else {} - cat(paste0("ggplot(",questions.frame,", aes(x=",questions.frame,"$",questions.name," , y=",questions.frame,"$",disag.name,")) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(",questions.frame,", aes(x=",questions.frame,"$",questions.name," , y=",questions.frame,"$",disag.name,")) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_boxplot(fill=\"#2a87c8\",colour=\"black\" ) + "),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("scale_size_area(max_size = 10) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("guides(fill=FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("guides(fill = FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("xlab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ylab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("coord_flip() +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("scale_y_continuous(breaks= pretty_breaks()) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("scale_y_continuous(breaks = pretty_breaks(), label = format_si()) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"",questions.label,"\","),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("subtitle = \"Before data capping treatement. By question: ",disag.label,".\") +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("theme(plot.title=element_text(face=\"bold\", size = 9 ),plot.background = element_rect(fill = \"transparent\",colour = NA))"),file = chapter.name ,sep = "\n", append = TRUE) - + cat(paste0("kobo_unhcr_style_histo()"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) ## Boxplot with capping treatment cat(paste0("## Boxplot"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("ggplot(",questions.frame,", aes(y=data.nooutlier1$variable, x= ",questions.frame,"$",questions.name,")) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(",questions.frame,", aes(y=data.nooutlier1$variable, x= ",questions.frame,"$",questions.name,")) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_boxplot(fill=\"#2a87c8\",colour=\"black\") + #notch=TRUE"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("scale_size_area(max_size = 10) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("guides(fill=FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("guides(fill = FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("xlab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ylab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("coord_flip() +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("scale_y_continuous(breaks= pretty_breaks()) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("scale_y_continuous(breaks = pretty_breaks(), label = format_si()) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"",questions.label,"\","),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("subtitle = \"After data capping treatement. By question: ",disag.label,".\") +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("theme(plot.title=element_text(face=\"bold\", size = 9 ),"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("plot.background = element_rect(fill = \"transparent\",colour = NA))"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("kobo_unhcr_style_histo()"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) ## Close chunk cat(paste0("\n```\n", sep = '\n'), file = chapter.name, append = TRUE) @@ -597,21 +625,23 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat(paste0("## and now the graph"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("ggplot(crosssfrequ.weight, aes(fill=crosssfrequ.weight$quest, y=crosssfrequ.weight$Freq, x = crosssfrequ.weight$disag)) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(crosssfrequ.weight, aes(fill=crosssfrequ.weight$quest, y=crosssfrequ.weight$Freq, x = crosssfrequ.weight$disag)) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_bar(colour=\"white\", stat =\"identity\", width=.8, aes(fill = quest), position = position_stack(reverse = TRUE)) +"),file = chapter.name ,sep = "\n", append = TRUE) #cat(paste0("geom_label_repel(aes(label = Freq2), fill = \"#2a87c8\", color = 'white') +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ylab(\"Frequency\") +"),file = chapter.name ,sep = "\n", append = TRUE) #cat(paste0("facet_wrap(~disag, ncol=3) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("scale_y_continuous(labels=percent) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("scale_y_continuous(labels = percent) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("scale_fill_viridis(discrete=TRUE) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("xlab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("coord_flip() +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"",questions.label," (color)\","),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("subtitle = \" By question: ",disag.label," (bar)\") +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("theme(plot.title=element_text(face=\"bold\", size = 9 ),plot.background = element_rect(fill = \"transparent\",colour = NA)) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("kobo_unhcr_style_bar() +"),file = chapter.name ,sep = "\n", append = TRUE) ## setting up the legend - #cat(paste0("guides(fill=FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + #cat(paste0("guides(fill = FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("theme(legend.direction = \"horizontal\", legend.position = \"bottom\", legend.box = \"horizontal\",legend.title=element_blank() )"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) + ## Close chunk cat(paste0("\n```\n", sep = ""), file = chapter.name, append = TRUE) cat("\n", file = chapter.name, append = TRUE) @@ -684,7 +714,7 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { ## Check that each class is represented check.class <- as.data.frame(table(formula$target,formula$tested)) n.class <- nrow(check.class) - n.class.notnull <- nrow(check.class[check.class$Freq>0, ]) + n.class.notnull <- nrow(check.class[check.class$Freq > 0, ]) ### Testing number of levels for the 2 variables as 'x' and 'y' must have at least 2 levels if ( (chiquare.result[1, c("target")] != chiquare.result[1, c("tested")] ) & @@ -772,10 +802,10 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat(paste0("cat(\"No responses recorded for this question...\")"),file = chapter.name , sep = "\n", append = TRUE) cat("No responses recorded for this question...\n") } else if (nrow(frequ) > 10) { - cat(paste0("cat(\"There's too many potential values to display. We will only show the histogram. \n \")"),file = chapter.name ,sep = "\n", append = TRUE) + # cat(paste0("cat(\"There's too many potential values to display. We will only show the histogram. \n \")"),file = chapter.name ,sep = "\n", append = TRUE) } else{ - cat(paste0("## display table"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("kable(frequ, caption=\"__Table__:", questions.label,"\")"),file = chapter.name ,sep = "\n", append = TRUE) + # cat(paste0("## display table"),file = chapter.name ,sep = "\n", append = TRUE) + # cat(paste0("kable(frequ, caption=\"__Table__:", questions.label,"\")"),file = chapter.name ,sep = "\n", append = TRUE) } ## To do implement FD number of bin: https://www.r-bloggers.com/friday-function-nclass/ @@ -786,13 +816,13 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat(paste0("average <- as.data.frame(svymean(~ ",questions.name,", design = ",questions.frame,".survey, na.rm = TRUE))"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("cat(paste0(\"Based on the sample design, the average weighted mean response for this question is \", as.numeric(round(average$mean, digits = 2))))"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("# regular histogram"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("ggplot(data = frequ, aes(x = frequ$Var1, y = frequ$Freq)) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(data = frequ, aes(x = frequ$Var1, y = frequ$Freq)) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_bar(fill = \"#2a87c8\",colour = \"white\", stat = \"identity\", width = .8) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("labs(x = \"\", y = \"Count\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"",questions.label,"\",subtitle = \"Before data capping treatement.\") +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0(""),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("kobo_unhcr_style_histo()"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("theme(plot.title = element_text(face=\"bold\", size = 9 ), plot.background = element_rect(fill = \"transparent\",colour = NA))"),file = chapter.name ,sep = "\n", append = TRUE) ### Detect outliers and adjust bien numbers ##### ### To -- check there's outlier or not @@ -808,13 +838,13 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { ### Now graphs with treated variable ##### - cat(paste0("ggplot(data = data.nooutlier, aes(x = data.nooutlier$variable)) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(data = data.nooutlier, aes(x = data.nooutlier$variable)) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_histogram(color = \"white\",fill = \"#2a87c8\", breaks = pretty(data.nooutlier$variable, n = nclass.Sturges(data.nooutlier$variable),min.n = 1)) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("labs(x = \"\", y = \"Count\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"",questions.label,"\","),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("subtitle = \"After data capping treatement.\") +"),file = chapter.name ,sep = "\n", append = TRUE) - - cat(paste0("theme(plot.title = element_text(face=\"bold\", size = 9), plot.background = element_rect(fill = \"transparent\",colour = NA))"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("kobo_unhcr_style_histo()"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) } ## Close chunk cat(paste0("\n```\n", sep = '\n'), file = chapter.name, append = TRUE) @@ -889,34 +919,33 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { ## Boxplot - cat(paste0("ggplot(",questions.frame,", aes(y=",questions.frame,"$",questions.name," , x=",questions.frame,"$",disag.name,")) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(",questions.frame,", aes(y=",questions.frame,"$",questions.name," , x=",questions.frame,"$",disag.name,")) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_boxplot(fill=\"#2a87c8\",colour=\"black\") + "),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("scale_size_area(max_size = 10) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("guides(fill=FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("guides(fill = FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("xlab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ylab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("coord_flip() +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("scale_y_continuous(breaks= pretty_breaks()) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"",questions.label,"\","),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("subtitle = \"Before data capping treatement, by question: ",disag.label,"\") +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("theme(plot.title=element_text(face=\"bold\", size = 9 ),"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("plot.background = element_rect(fill = \"transparent\",colour = NA))"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("kobo_unhcr_style_bar()"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) ## Boxplot with capping treatment cat(paste0("## Boxplot"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("ggplot(",questions.frame,", aes(y=data.nooutlier$variable, x= ",questions.frame,"$",disag.name,")) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(",questions.frame,", aes(y=data.nooutlier$variable, x= ",questions.frame,"$",disag.name,")) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_boxplot(fill=\"#2a87c8\",colour=\"black\") + #notch=TRUE"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("scale_size_area(max_size = 10) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("guides(fill=FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("guides(fill = FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("xlab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ylab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("coord_flip() +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("scale_y_continuous(breaks= pretty_breaks()) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"",questions.label,"\","),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("subtitle = \"After data capping treatement. By question: ",disag.label,"\") +"),file = chapter.name ,sep = "\n", append = TRUE) - - cat(paste0("theme(plot.title=element_text(face=\"bold\", size = 9 ),"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("plot.background = element_rect(fill = \"transparent\",colour = NA))"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("kobo_unhcr_style_bar()"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) ## Close chunk cat(paste0("\n```\n", sep = '\n'), file = chapter.name, append = TRUE) @@ -940,30 +969,34 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat(paste0("names(data.nooutlier1)[1] <- \"variable\""),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("## Scatter plot"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("ggplot(",questions.frame,", aes(x= ",questions.frame,"$",disag.name, ", y=",questions.frame,"$",questions.name,")) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(",questions.frame,", aes(x= ",questions.frame,"$",disag.name, ", y=",questions.frame,"$",questions.name,")) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_count(aes(size = ..prop.., group = 1)) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("scale_size_area(max_size = 10) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("guides(fill=FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("guides(fill = FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("scale_y_continuous(breaks = pretty_breaks(), label = format_si()) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("scale_x_continuous(breaks = pretty_breaks(), label = format_si()) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("# xlab(correllabel) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("#ylab(variablelabel) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("# ylab(variablelabel) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_smooth(method=lm) + # Add a loess smoothed fit curve with confidence region"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"Scatterplot before data capping treatment\") +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("theme(plot.title=element_text(face=\"bold\", size = 9 ),"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("plot.background = element_rect(fill = \"transparent\",colour = NA))"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("kobo_unhcr_style_scatter()"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("## Scatter plot rev "),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("ggplot(",questions.frame,", aes(x= data.nooutlier$variable, y=data.nooutlier1$variable )) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(",questions.frame,", aes(x= data.nooutlier$variable, y=data.nooutlier1$variable )) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_count(aes(size = ..prop.., group = 1)) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("scale_size_area(max_size = 10) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("guides(fill=FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("guides(fill = FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("scale_y_continuous(breaks = pretty_breaks(), label = format_si()) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("scale_x_continuous(breaks = pretty_breaks(), label = format_si()) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("# xlab(correllabel) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("#ylab(variablelabel) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_smooth(method=lm) + # Add a loess smoothed fit curve with confidence region"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"Scatterplot after data capping treatment\") +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("theme(plot.title=element_text(face=\"bold\", size = 9 ),"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("plot.background = element_rect(fill = \"transparent\",colour = NA))"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("kobo_unhcr_style_scatter()"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) ## Close chunk cat(paste0("\n```\n", sep = '\n'), file = chapter.name, append = TRUE) @@ -1031,7 +1064,7 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat(paste0("check <- as.data.frame(names(",questions.frame ,"))"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("names(check)[1] <- \"check\""),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("check$id <- row.names(check)"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("check <- merge(x=check, y=selectmultilist1,by=\"check\")"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("check <- merge(x = check, y = selectmultilist1, by = \"check\")"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("selectmultilist <- as.character(check[ ,1])"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("## Reshape answers"),file = chapter.name ,sep = "\n", append = TRUE) @@ -1044,14 +1077,14 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat(paste0("castdata <- as.data.frame(table(meltdata[c(\"value\")]))"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("castdata$freqper <- castdata$Freq/nrow(data.selectmultilist)"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("castdata <- castdata[castdata$Var1!=\"Not selected\", ]"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("castdata$Var1 <-factor(castdata$Var1, levels=castdata[order(castdata$freqper), \"Var1\"])"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("castdata$Var1 <- factor(castdata$Var1, levels=castdata[order(castdata$freqper), \"Var1\"])"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("frequ <- castdata[castdata$Var1!=\"\", ]"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("## display table"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("names(frequ)[1] <- \"", questions.shortname,"\""),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("frequ[ ,3] <- paste0(round(frequ[ ,3]*100,digits = 1),\"%\")"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("kable(frequ, caption=\"__Table__:", questions.label,"\")"),file = chapter.name ,sep = "\n", append = TRUE) + # cat(paste0("kable(frequ, caption=\"__Table__:", questions.label,"\")"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("frequ1 <- castdata[castdata$Var1!=\"\", ]"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("frequ1[ ,4] <- paste0(round(frequ1[ ,3]*100,digits = 1),\"%\")"),file = chapter.name ,sep = "\n", append = TRUE) @@ -1060,18 +1093,18 @@ kobo_crunching_report <- function(form = "form.xls", app="console") { cat(paste0("\n"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("## and now the graph"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("ggplot(frequ1, aes(x=Var1, y=freqper)) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("geom_bar(fill=\"#2a87c8\",colour=\"#2a87c8\", stat =\"identity\", width=.8) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("guides(fill=FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("plot1 <- ggplot(frequ1, aes(x=Var1, y=freqper)) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("geom_bar(fill = \"#2a87c8\", colour = \"#2a87c8\", stat = \"identity\", width=.8) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("guides(fill = FALSE) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("geom_label_repel(aes(y = freqper, label = freqper2), fill = \"#2a87c8\", color = 'white') +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ylab(\"Frequency\") +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("scale_y_continuous(labels=percent) +"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("scale_y_continuous(labels = percent) +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("xlab(\"\") +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("coord_flip() +"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("ggtitle(\"",questions.label,"\","),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("subtitle = paste0(\"Question response rate: \",percentreponse,\" .\")) +"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("theme(plot.title=element_text(face=\"bold\", size = 9 ),"),file = chapter.name ,sep = "\n", append = TRUE) - cat(paste0("plot.background = element_rect(fill = \"transparent\",colour = NA))"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("kobo_unhcr_style_bar()"),file = chapter.name ,sep = "\n", append = TRUE) + cat(paste0("ggpubr::ggarrange(kobo_left_align(plot1, c(\"subtitle\", \"title\")), ncol = 1, nrow = 1)"),file = chapter.name ,sep = "\n", append = TRUE) cat(paste0("\n```\n", sep = '\n'), file = chapter.name, append = TRUE) ###select.multi.rel###################################################################### diff --git a/R/kobo_dico.R b/R/kobo_dico.R index 4cd88f9..abe8a85 100644 --- a/R/kobo_dico.R +++ b/R/kobo_dico.R @@ -39,6 +39,12 @@ kobo_dico <- function(form = "form.xls") { ## Rename the variable label names(survey)[names(survey) == "label::English"] <- "label" names(survey)[names(survey) == "label::english"] <- "label" + + + names(survey)[names(survey) == "hint::English"] <- "hint" + names(survey)[names(survey) == "hint::english"] <- "hint" + + cat("Checking now for additional information within your xlsform. Note that you can insert them in the xls and re-run the function! \n \n ") @@ -46,134 +52,142 @@ kobo_dico <- function(form = "form.xls") { ### add column if not present ################################################# if ("labelReport" %in% colnames(survey)) { - cat("1- Good: You have a column `labelReport` in your survey worksheet.\n"); + cat(" Good: You have a column `labelReport` in your survey worksheet.\n"); } else - {cat("1- No column `labelReport` in your survey worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `labelReport` in your survey worksheet. Creating a dummy one for the moment...\n"); survey[,"labelReport"] <- substr(survey[,"label"],1,80)} + + if ("hintReport" %in% colnames(survey)) + { + cat(" Good: You have a column `hintReport` in your survey worksheet.\n"); + } else + {cat(" No column `hintReport` in your survey worksheet. Creating a dummy one for the moment...\n"); + survey[,"hintReport"] <- survey[,"hint"]} + if ("disaggregation" %in% colnames(survey)) { - cat("1- Good: You have a column `disaggregation` in your survey worksheet.\n"); + cat(" Good: You have a column `disaggregation` in your survey worksheet.\n"); } else - {cat("1- No column `disaggregation` in your survey worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `disaggregation` in your survey worksheet. Creating a dummy one for the moment...\n"); survey$disaggregation <- ""} if ("correlate" %in% colnames(survey)) { - cat("2- Good: You have a column `correlate` in your survey worksheet. This will be used to define the variables that should be checked for correlation between each others.\n"); + cat(" Good: You have a column `correlate` in your survey worksheet. This will be used to define the variables that should be checked for correlation between each others.\n"); } else - {cat("2- No column `correlate` in your survey worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `correlate` in your survey worksheet. Creating a dummy one for the moment...\n"); survey$correlate <- ""} if ("chapter" %in% colnames(survey)) { - cat("3- Good: You have a column `chapter` in your survey worksheet. This will be used to breakdown the generated report\n"); + cat(" Good: You have a column `chapter` in your survey worksheet. This will be used to breakdown the generated report\n"); } else - {cat("3- No column `chapter` in your survey worksheet. Creating a dummy one for the moment ...\n"); + {cat(" No column `chapter` in your survey worksheet. Creating a dummy one for the moment ...\n"); survey$chapter <- ""} if ("structuralequation.risk" %in% colnames(survey)) { - cat("4- Good: You have a column `structuralequation.risk` in your survey worksheet. This will be used to configure the vulnerability structural equation model\n"); + cat(" Good: You have a column `structuralequation.risk` in your survey worksheet. This will be used to configure the vulnerability structural equation model\n"); } else - {cat("4- No column `structuralequation.risk` in your survey worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `structuralequation.risk` in your survey worksheet. Creating a dummy one for the moment...\n"); survey$structuralequation.risk <- ""} if ("structuralequation.coping" %in% colnames(survey)) { - cat("4- Good: You have a column `structuralequation.coping` in your survey worksheet. This will be used to configure the vulnerability structural equation model\n"); + cat(" Good: You have a column `structuralequation.coping` in your survey worksheet. This will be used to configure the vulnerability structural equation model\n"); } else {cat("4- No column `structuralequation.coping` in your survey worksheet. Creating a dummy one for the moment...\n"); survey$structuralequation.coping <- ""} if ("structuralequation.resilience" %in% colnames(survey)) { - cat("4- Good: You have a column `structuralequation.resilience` in your survey worksheet. This will be used to configure the vulnerability structural equation model\n"); + cat(" Good: You have a column `structuralequation.resilience` in your survey worksheet. This will be used to configure the vulnerability structural equation model\n"); } else - {cat("4- No column `structuralequation.resilience` in your survey worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `structuralequation.resilience` in your survey worksheet. Creating a dummy one for the moment...\n"); survey$structuralequation.resilience <- ""} if ("anonymise" %in% colnames(survey)) { - cat("5- Good: You have a column `anonymise` in your survey worksheet. This will be used to anonymise the dataset.\n"); + cat(" Good: You have a column `anonymise` in your survey worksheet. This will be used to anonymise the dataset.\n"); } else - {cat("5- No column `anonymise` in your survey worksheet. Creating a dummy one for the moment filled as `non-anonymised`. Other options to record are `Remove`, `Reference`, `Mask`, `Generalise` (see readme file) ...\n"); + {cat(" No column `anonymise` in your survey worksheet. Creating a dummy one for the moment filled as `non-anonymised`. Other options to record are `Remove`, `Reference`, `Mask`, `Generalise` (see readme file) ...\n"); survey$anonymise <- "default-non-anonymised"} if ("variable" %in% colnames(survey)) { - cat("6- Good: You have a column `variable` in your survey worksheet. This will be used to flag ordinal variable.\n"); + cat(" Good: You have a column `variable` in your survey worksheet. This will be used to flag ordinal variable.\n"); } else - {cat("6- No column `variable` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); + {cat(" No column `variable` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); survey$variable <- ""} ## Adding clean cluster predict if ("clean" %in% colnames(survey)) { - cat("7- Good: You have a column `clean` in your survey worksheet. This will be used to flag variables that shoudl be clean with kobo_clean function.\n"); + cat(" Good: You have a column `clean` in your survey worksheet. This will be used to flag variables that shoudl be clean with kobo_clean function.\n"); } else - {cat("7- No column `clean` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); + {cat(" No column `clean` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); survey$clean <- "no"} if ("cluster" %in% colnames(survey)) { - cat("8- Good: You have a column `cluster` in your survey worksheet. This will be used to flag variables to be used for clustering exploration.\n"); + cat(" Good: You have a column `cluster` in your survey worksheet. This will be used to flag variables to be used for clustering exploration.\n"); } else - {cat("8- No column `cluster` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); + {cat(" No column `cluster` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); survey$cluster <- ""} if ("predict" %in% colnames(survey)) { - cat("9- Good: You have a column `predict` in your survey worksheet. This will be used to flag variables to be used for clustering exploration.\n"); + cat(" Good: You have a column `predict` in your survey worksheet. This will be used to flag variables to be used for clustering exploration.\n"); } else - {cat("9- No column `predict` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); + {cat(" No column `predict` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); survey$predict <- ""} if ("mappoint" %in% colnames(survey)) { - cat("10- Good: You have a column `mappoint` in your survey worksheet. This will be used to flag variables to be used for clustering exploration.\n"); + cat(" Good: You have a column `mappoint` in your survey worksheet. This will be used to flag variables to be used for clustering exploration.\n"); } else - {cat("10- No column `mappoint` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); + {cat(" No column `mappoint` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); survey$mappoint <- ""} if ("mappoly" %in% colnames(survey)) { - cat("11- Good: You have a column `mappoly` in your survey worksheet. This will be used to flag variables to be used for clustering exploration.\n"); + cat(" Good: You have a column `mappoly` in your survey worksheet. This will be used to flag variables to be used for clustering exploration.\n"); } else - {cat("11- No column `mappoly` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); + {cat(" No column `mappoly` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); survey$mappoly <- ""} if ("relevant" %in% colnames(survey)) { - cat("1- Good: You have a column `relevant` in your survey worksheet.\n"); + cat(" Good: You have a column `relevant` in your survey worksheet.\n"); } else - {cat("1- No column `relevant` in your survey worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `relevant` in your survey worksheet. Creating a dummy one for the moment...\n"); survey[,"relevant"] <- ""} - + if ("required" %in% colnames(survey)) { - cat("1- Good: You have a column `required` in your survey worksheet.\n"); + cat(" Good: You have a column `required` in your survey worksheet.\n"); } else - {cat("1- No column `required` in your survey worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `required` in your survey worksheet. Creating a dummy one for the moment...\n"); survey[,"required"] <- ""} - + if ("constraint" %in% colnames(survey)) { - cat("1- Good: You have a column `constraint` in your survey worksheet.\n"); + cat(" Good: You have a column `constraint` in your survey worksheet.\n"); } else - {cat("1- No column `constraint` in your survey worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `constraint` in your survey worksheet. Creating a dummy one for the moment...\n"); survey[,"constraint"] <- ""} - + if ("repeat_count" %in% colnames(survey)) { - cat("1- Good: You have a column `repeat_count` in your survey worksheet.\n"); + cat(" Good: You have a column `repeat_count` in your survey worksheet.\n"); } else - {cat("1- No column `repeat_count` in your survey worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `repeat_count` in your survey worksheet. Creating a dummy one for the moment...\n"); survey[,"repeat_count"] <- ""} - + ## Avoid columns without names - survey <- survey[ ,c("type", "name" , "label", "labelReport", + survey <- survey[ ,c("type", "name" , "label", "labelReport", "hintReport", #"repeatsummarize", "variable","disaggregation", "chapter", "structuralequation.risk","structuralequation.coping","structuralequation.resilience", "anonymise","correlate","clean","cluster","predict","mappoint","mappoly", @@ -249,7 +263,9 @@ kobo_dico <- function(form = "form.xls") { } ### identify Repeat questions - survey$qrepeatlabel <- "MainDataFrame_edited" + + survey$qrepeatlabel <- "MainDataFrame" + nestable <- survey[survey$type %in% c("begin_repeat","begin repeat") , c("name","qrepeat","type")] nestable$name <- as.character(nestable$name) for (i in 2:nrow(survey)) @@ -259,7 +275,9 @@ kobo_dico <- function(form = "form.xls") { if ( survey[ i, c("type")] == "begin repeat" ) {survey[ i, c("qrepeatlabel")] <- survey[ i, c("name")]} else if ( survey[ i, c("type")] != "end repeat" && survey[ i - 1, c("qrepeat")] == "repeatnest1" ) {survey[ i, c("qrepeatlabel")] <- survey[ i - 1, c("qrepeatlabel")] } else if ( survey[ i, c("type")] != "end repeat" && survey[ i - 1, c("qrepeat")] == "repeatnest2" ) {survey[ i, c("qrepeatlabel")] <- survey[ i - 1, c("qrepeatlabel")] } - else if ( survey[ i, c("type")] == "end repeat" && survey[ i - 1, c("qrepeat")] == "repeatnest1") {survey[ i, c("qrepeatlabel")] <- "MainDataFrame_edited"} + + else if ( survey[ i, c("type")] == "end repeat" && survey[ i - 1, c("qrepeat")] == "repeatnest1") {survey[ i, c("qrepeatlabel")] <- "MainDataFrame"} + else if ( survey[ i, c("type")] == "end repeat" && survey[ i - 1, c("qrepeat")] == "repeatnest2") { nestabove <- as.character(survey[ i - 1, c("qrepeatlabel")]) nestabovenum <- as.integer(which(nestable$name == nestabove ) - 1) survey[ i, c("qrepeatlabel")] <- as.character( nestable[ nestabovenum , 1] ) } @@ -268,12 +286,16 @@ kobo_dico <- function(form = "form.xls") { else if ( survey[ i, c("type")] == "begin_repeat" ) {survey[ i, c("qrepeatlabel")] <- survey[ i, c("name")]} else if ( survey[ i, c("type")] != "end_repeat" && survey[ i - 1, c("qrepeat")] == "repeatnest1" ) {survey[ i, c("qrepeatlabel")] <- survey[ i - 1, c("qrepeatlabel")] } else if ( survey[ i, c("type")] != "end_repeat" && survey[ i - 1, c("qrepeat")] == "repeatnest2" ) {survey[ i, c("qrepeatlabel")] <- survey[ i - 1, c("qrepeatlabel")] } - else if ( survey[ i, c("type")] == "end_repeat" && survey[ i - 1, c("qrepeat")] == "repeatnest1") {survey[ i, c("qrepeatlabel")] <- "MainDataFrame_edited"} + + else if ( survey[ i, c("type")] == "end_repeat" && survey[ i - 1, c("qrepeat")] == "repeatnest1") {survey[ i, c("qrepeatlabel")] <- "MainDataFrame"} + else if ( survey[ i, c("type")] == "end_repeat" && survey[ i - 1, c("qrepeat")] == "repeatnest2") { nestabove <- as.character(survey[ i - 1, c("qrepeatlabel")]) nestabovenum <- as.integer(which(nestable$name == nestabove ) - 1) survey[ i, c("qrepeatlabel")] <- as.character( nestable[ nestabovenum , 1] ) } - else {survey[ i, c("qrepeatlabel")] <- "MainDataFrame_edited"} + + else {survey[ i, c("qrepeatlabel")] <- "MainDataFrame"} + } ### Get question levels in order to match the variable name @@ -400,41 +422,45 @@ kobo_dico <- function(form = "form.xls") { if ("labelReport" %in% colnames(choices)) { - cat("12 - Good: You have a column `labelReport` in your `choices` worksheet.\n"); + cat(" Good: You have a column `labelReport` in your `choices` worksheet.\n"); } else - {cat("12 - No column `labelReport` in your `choices` worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `labelReport` in your `choices` worksheet. Creating a dummy one for the moment...\n"); choices[,"labelReport"] <- substr(choices[,"label"],1,80)} if ("order" %in% colnames(choices)) { - cat("12 - Good: You have a column `order` in your `choices` worksheet.\n"); + cat(" Good: You have a column `order` in your `choices` worksheet.\n"); } else - {cat("12 - No column `order` in your `choices` worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `order` in your `choices` worksheet. Creating a dummy one for the moment...\n"); choices$order <- ""} if ("weight" %in% colnames(choices)) { - cat("13 - Good: You have a column `weight` in your `choices` worksheet.\n"); + cat(" Good: You have a column `weight` in your `choices` worksheet.\n"); } else - {cat("13 - No column `weight` in your `choices` worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `weight` in your `choices` worksheet. Creating a dummy one for the moment...\n"); choices$weight <- ""} if ("recategorise" %in% colnames(choices)) { - cat("14 - Good: You have a column `recategorise` in your `choices` worksheet.\n"); + cat(" Good: You have a column `recategorise` in your `choices` worksheet.\n"); } else - {cat("14 - No column `recategorise` in your `choices` worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `recategorise` in your `choices` worksheet. Creating a dummy one for the moment...\n"); choices$recategorise <- ""} if ("score" %in% colnames(choices)) { - cat("13 - Good: You have a column `score` in your `choices` worksheet.\n"); + cat(" Good: You have a column `score` in your `choices` worksheet.\n"); } else - {cat("13 - No column `score` in your `choices` worksheet. Creating a dummy one for the moment...\n"); + {cat(" No column `score` in your `choices` worksheet. Creating a dummy one for the moment...\n"); choices$score <- ""} choices <- choices[,c("listname", "name", "labelReport", "order", "weight","score","recategorise")] + + + + names(choices)[names(choices) == "labelReport"] <- "labelchoice" #rm(choices) choices <- join(x = choices, y = survey, by = "listname", type = "left") @@ -458,7 +484,7 @@ kobo_dico <- function(form = "form.xls") { #names(choices) -"type", "name", "namefull", "labelfull", "listname", "qrepeat", "qlevel", "qgroup" ## not kept: "nameq" "labelq" ,"fullname", "label", #names(survey) - "type" "name", "fullname", "label", "listname", "qrepeat"m "qlevel", "qgroup" - choices2 <- choices[ ,c("type", "name", "namefull", "labelfull", "labelReport", "chapter","disaggregation","correlate", "structuralequation.risk","structuralequation.coping","structuralequation.resilience","anonymise", + choices2 <- choices[ ,c("type", "name", "namefull", "labelfull", "labelReport","hintReport", "chapter","disaggregation","correlate", "structuralequation.risk","structuralequation.coping","structuralequation.resilience","anonymise", "clean","cluster","predict","mappoint","mappoly", "relevant", "required", "constraint", "repeat_count", "listname", "qrepeat","qrepeatlabel", "qlevel", "qgroup", "labelchoice", @@ -472,7 +498,7 @@ kobo_dico <- function(form = "form.xls") { names(choices2)[names(choices2) == "labelfull"] <- "label" - survey2 <- survey[,c("type", "name", "fullname", "label", "labelReport", "chapter", "disaggregation","correlate", "structuralequation.risk","structuralequation.coping","structuralequation.resilience","anonymise", + survey2 <- survey[,c("type", "name", "fullname", "label", "labelReport","hintReport", "chapter", "disaggregation","correlate", "structuralequation.risk","structuralequation.coping","structuralequation.resilience","anonymise", "clean","cluster","predict","mappoint","mappoly", "relevant", "required", "constraint", "repeat_count", "listname", "qrepeat","qrepeatlabel", "qlevel", "qgroup", "labelchoice", diff --git a/R/kobo_dummy.R b/R/kobo_dummy.R index 1309140..9c1ed1c 100644 --- a/R/kobo_dummy.R +++ b/R/kobo_dummy.R @@ -33,9 +33,9 @@ kobo_dummy <- function(form = "form.xls") { - ### Write dummy dataset - + samplesize <- 381 + ### Write dummy dataset #kobodevtools::install_github("ropensci/charlatan") #devtools::install_github("ThinkR-open/fakir") # install.packages("truncnorm") @@ -181,7 +181,7 @@ kobo_dummy <- function(form = "form.xls") { ## Create corresponding dummy data ######## - samplesize <- 500 + ## generate the unique ID for each observation dummydata <- data.frame(stri_rand_strings(samplesize, 8)) @@ -321,12 +321,12 @@ kobo_dummy <- function(form = "form.xls") { #levels(as.factor(as.character(dico.repeat$type))) for (h in 1:length(repeat_name)) { - # h <-2 + # h <- 2 repeat_table <- as.character(repeat_name[h]) ## Build corresponding repeat frame dico.repeat1 <- dico.repeat[dico.repeat$qrepeatlabel == repeat_table, ] - ## Getting records to be generated for each ID + cat("Getting records to be generated for each ID \n\n\n") maxvariable <- as.character(dico[dico$qrepeatlabel == repeat_table & dico$type %in% c("begin_repeat", "begin repeat") , c("repeat_count") ]) @@ -335,140 +335,152 @@ kobo_dummy <- function(form = "form.xls") { maxvariablefullname <- maxvariablefullname[!(is.na(maxvariablefullname$fullname)), c("fullname")] maxvariablefullname <- as.character(maxvariablefullname) #str(maxvariablefullname) - + rm(dummydatamaxvariable) dummydatamaxvariable <- dummydata[ ,c("instanceID",maxvariablefullname )] #str(dummydatamaxvariable) ## Account for NA - relevant nested table dummydatamaxvariable <- dummydatamaxvariable[ !(is.na(dummydatamaxvariable[ ,2])), ] - - - # names(dummydata) - + # names(dummydata) #dummydatarepeat <- data.frame("instanceID" ) #names(dummydatarepeat)[1] <- "instanceID" dummydatarepeatall <- as.data.frame(matrix(0, ncol = 1 + nrow(dico.repeat1), nrow = 0)) names(dummydatarepeatall)[1] <- "instanceID" names(dummydatarepeatall)[2:(nrow(dico.repeat1) + 1)] <- as.character(dico.repeat1[ ,c("fullname")]) - ## Loop around IDs - for (j in 1:nrow(dummydatamaxvariable) ) { - # j <- 1 - samplesize <- as.numeric(dummydatamaxvariable[ j, 2]) - this.id <- as.character(dummydatamaxvariable[ j, 1]) - - dummydatarepeat <- as.data.frame(matrix(0, ncol = 1, nrow = samplesize)) - dummydatarepeat[1] <- this.id - names(dummydatarepeat)[1] <- "instanceID" - - ## Loop around variables - for (i in 1:nrow(dico.repeat1) ) { - # i <- 46 - fullname <- as.character(dico.repeat1[i, c("fullname")]) - typedata <- as.character(dico.repeat1[dico.repeat1$fullname == fullname, c("type")]) - - - relevantifvar <- as.character(dico.repeat1[dico.repeat1$fullname == fullname, c("relevantifvar")]) - relevantifvar2 <- as.character(dico.repeat1[dico.repeat1$name == relevantifvar, c("fullname")]) - relevantifvalue <- as.character(dico.repeat1[dico.repeat1$fullname == fullname, c("relevantifvalue")]) - - cat(paste0("Entering dummy data for nested table ", h, " - ", repeat_table, - "for case ", j, - " for variable ", i, "- ", fullname, " / ", typedata,"\n")) - if (typedata %in% c("date") ) { - dummydatarepeat[ , i + 1] <- sample(seq(as.Date('2017/01/01'), as.Date('2019/01/01'), by = "day"), - replace = TRUE, - size = samplesize) - } - - if (typedata == "select_one") { - listname <- as.character(dico[dico$fullname == fullname & - dico$type == "select_one", c("listname")]) - categ_level <- as.character( unique(dico[dico$listname == listname & - dico$type == "select_one_d", c("name")])) - dummydatarepeat[ , i + 1] <- factor(sample(categ_level, - size = samplesize, - replace = TRUE)) - } - if (typedata == "select_multiple_d") { - listname <- as.character(dico[dico$fullname == fullname & - dico$type == "select_multiple_d", c("listname")]) - categ_level <- as.character( unique(dico[dico$listname == listname & - dico$type == "select_multiple", c("name")])) - dummydatarepeat[ , i + 1] <- factor(sample(categ_level, - size = samplesize, - replace = TRUE)) - } - if (typedata == "integer") { - lowerbound <- ifelse( is.na(as.numeric(dico.repeat1[ i, c("lowerbound")])), 0, as.numeric(dico.repeat1[ i, c("lowerbound")])) - upperbound <- ifelse(is.na(as.numeric(dico.repeat1[ i, c("upperbound")])), 100, as.numeric(dico.repeat1[ i, c("upperbound")])) - dummydatarepeat[ , i + 1] <- round(rtruncnorm(n = samplesize, - a = lowerbound, #lowerbound, # vector of lower bounds. These may be -Inf - b = upperbound, # vector of upper bounds. These may be Inf - mean = ((upperbound - lowerbound ) / 2), # vector of means. - sd = ((upperbound - lowerbound ) / 4) # vector of standard deviations. - )) - } - if (typedata == "calculate") { - lowerbound <- ifelse( is.na(as.numeric(dico.repeat1[ i, c("lowerbound")])), 0, as.numeric(dico.repeat1[ i, c("lowerbound")])) - upperbound <- ifelse(is.na(as.numeric(dico.repeat1[ i, c("upperbound")])), 100, as.numeric(dico.repeat1[ i, c("upperbound")])) - dummydatarepeat[ , i + 1] <- round(rtruncnorm(n = samplesize, - a = lowerbound, #lowerbound, # vector of lower bounds. These may be -Inf - b = upperbound, # vector of upper bounds. These may be Inf - mean = ((upperbound - lowerbound ) / 2), # vector of means. - sd = ((upperbound - lowerbound ) / 4) # vector of standard deviations. - )) - } - if (typedata == "decimal") { - lowerbound <- ifelse( is.na(as.numeric(dico.repeat1[ i, c("lowerbound")])), 0, as.numeric(dico.repeat1[ i, c("lowerbound")])) - upperbound <- ifelse(is.na(as.numeric(dico.repeat1[ i, c("upperbound")])), 100, as.numeric(dico.repeat1[ i, c("upperbound")])) - dummydatarepeat[ , i + 1] <- rtruncnorm(n = samplesize, - a = lowerbound, #lowerbound, # vector of lower bounds. These may be -Inf - b = upperbound, # vector of upper bounds. These may be Inf - mean = ((upperbound - lowerbound ) / 2), # vector of means. - sd = ((upperbound - lowerbound ) / 4) # vector of standard deviations. - ) - } - - if (typedata == "text") { - #dummydatarepeat[ , i + 1] <- "this is a dummy text" - dummydatarepeat[ , i + 1] <- randomSentences(n = samplesize, 3:10) - } - - ## Then rename correctly - names(dummydatarepeat)[i + 1 ] <- fullname - #cat(summary(dummydatarepeat[i])) + # if ( nrow(dico.repeat1) == 1) { + # ## only one variable linked on the second table + # + # + # } else { + + + ## Loop around IDs for each case + for (j in 1:nrow(dummydatamaxvariable) ) { + # j <- 1 + samplesize <- as.numeric(dummydatamaxvariable[ j, 2]) + + if (samplesize !=0 ) { + this.id <- as.character(dummydatamaxvariable[ j, 1]) + + dummydatarepeat <- as.data.frame(matrix(0, ncol = 1, nrow = samplesize)) + dummydatarepeat[1] <- this.id + names(dummydatarepeat)[1] <- "instanceID" + + + + ## Loop around variables + for (i in 1:nrow(dico.repeat1) ) { + # i <- 1 + fullname <- as.character(dico.repeat1[i, c("fullname")]) + typedata <- as.character(dico.repeat1[dico.repeat1$fullname == fullname, c("type")]) + + + relevantifvar <- as.character(dico.repeat1[dico.repeat1$fullname == fullname, c("relevantifvar")]) + relevantifvar2 <- as.character(dico.repeat1[dico.repeat1$name == relevantifvar, c("fullname")]) + relevantifvalue <- as.character(dico.repeat1[dico.repeat1$fullname == fullname, c("relevantifvalue")]) + + cat(paste0("Entering dummy data for nested table ", h, " - ", repeat_table, + "for case ", j, + " for variable ", i, "- ", fullname, " / ", typedata,"\n")) + if (typedata %in% c("date") ) { + dummydatarepeat[ , i + 1] <- sample(seq(as.Date('2017/01/01'), as.Date('2019/01/01'), by = "day"), + replace = TRUE, + size = samplesize) + } + + if (typedata == "select_one") { + listname <- as.character(dico[dico$fullname == fullname & + dico$type == "select_one", c("listname")]) + categ_level <- as.character( unique(dico[dico$listname == listname & + dico$type == "select_one_d", c("name")])) + dummydatarepeat[ , i + 1] <- factor(sample(categ_level, + size = samplesize, + replace = TRUE)) + } + if (typedata == "select_multiple_d") { + listname <- as.character(dico[dico$fullname == fullname & + dico$type == "select_multiple_d", c("listname")]) + categ_level <- as.character( unique(dico[dico$listname == listname & + dico$type == "select_multiple", c("name")])) + dummydatarepeat[ , i + 1] <- factor(sample(categ_level, + size = samplesize, + replace = TRUE)) + } + if (typedata == "integer") { + lowerbound <- ifelse( is.na(as.numeric(dico.repeat1[ i, c("lowerbound")])), 0, as.numeric(dico.repeat1[ i, c("lowerbound")])) + upperbound <- ifelse(is.na(as.numeric(dico.repeat1[ i, c("upperbound")])), 100, as.numeric(dico.repeat1[ i, c("upperbound")])) + dummydatarepeat[ , i + 1] <- round(rtruncnorm(n = samplesize, + a = lowerbound, #lowerbound, # vector of lower bounds. These may be -Inf + b = upperbound, # vector of upper bounds. These may be Inf + mean = ((upperbound - lowerbound ) / 2), # vector of means. + sd = ((upperbound - lowerbound ) / 4) # vector of standard deviations. + )) + } + if (typedata == "calculate") { + lowerbound <- ifelse( is.na(as.numeric(dico.repeat1[ i, c("lowerbound")])), 0, as.numeric(dico.repeat1[ i, c("lowerbound")])) + upperbound <- ifelse(is.na(as.numeric(dico.repeat1[ i, c("upperbound")])), 100, as.numeric(dico.repeat1[ i, c("upperbound")])) + dummydatarepeat[ , i + 1] <- round(rtruncnorm(n = samplesize, + a = lowerbound, #lowerbound, # vector of lower bounds. These may be -Inf + b = upperbound, # vector of upper bounds. These may be Inf + mean = ((upperbound - lowerbound ) / 2), # vector of means. + sd = ((upperbound - lowerbound ) / 4) # vector of standard deviations. + )) + } + if (typedata == "decimal") { + lowerbound <- ifelse( is.na(as.numeric(dico.repeat1[ i, c("lowerbound")])), 0, as.numeric(dico.repeat1[ i, c("lowerbound")])) + upperbound <- ifelse(is.na(as.numeric(dico.repeat1[ i, c("upperbound")])), 100, as.numeric(dico.repeat1[ i, c("upperbound")])) + dummydatarepeat[ , i + 1] <- rtruncnorm(n = samplesize, + a = lowerbound, #lowerbound, # vector of lower bounds. These may be -Inf + b = upperbound, # vector of upper bounds. These may be Inf + mean = ((upperbound - lowerbound ) / 2), # vector of means. + sd = ((upperbound - lowerbound ) / 4) # vector of standard deviations. + ) + } + + if (typedata == "text") { + #dummydatarepeat[ , i + 1] <- "this is a dummy text" + dummydatarepeat[ , i + 1] <- randomSentences(n = samplesize, 3:10) + } + + ## Then rename correctly + names(dummydatarepeat)[i + 1 ] <- fullname + #cat(summary(dummydatarepeat[i])) + + + ## Put to NA if relevance condition is set and not respected + if ( !(is.na(relevantifvar)) & relevantifvar != "" ) { + datacheck <- as.data.frame(dummydatarepeat[ , c(relevantifvar2) ]) + cat(paste0(" Apply relevance on ",relevantifvar2," \n")) + for (l in 1:nrow(dummydatarepeat)) { + # l <- 3 + + value <- ifelse(is.na(datacheck[l,]),"", + ifelse(datacheck[l,] == relevantifvalue , paste(dummydatarepeat[l ,i + 1 ]), "")) + if (value == "") { + dummydatarepeat[l ,i + 1 ] <- NA } else { + dummydatarepeat[l ,i + 1 ] <- value + } + #dummydatarepeat[l ,i + 1 ] <- ifelse(datacheck[l,] == relevantifvalue , paste(dummydatarepeat[l ,i + 1 ]), "") + } + } - ## Put to NA if relevance condition is set and not respected - if ( !(is.na(relevantifvar)) & relevantifvar != "" ) { - datacheck <- as.data.frame(dummydatarepeat[ , c(relevantifvar2) ]) - cat(paste0(" Apply relevance on ",relevantifvar2," \n")) - for (l in 1:nrow(dummydatarepeat)) { - # l <- 3 - value <- ifelse(is.na(datacheck[l,]),"", - ifelse(datacheck[l,] == relevantifvalue , paste(dummydatarepeat[l ,i + 1 ]), "")) - if (value == "") { - dummydatarepeat[l ,i + 1 ] <- NA } else { - dummydatarepeat[l ,i + 1 ] <- value - } - #dummydatarepeat[l ,i + 1 ] <- ifelse(datacheck[l,] == relevantifvalue , paste(dummydatarepeat[l ,i + 1 ]), "") + } + cat("Appending this record \n\n") + dummydatarepeatall <- rbind(dummydatarepeatall, dummydatarepeat) + rm(dummydatarepeat) } } - - - } - dummydatarepeatall <- rbind(dummydatarepeatall, dummydatarepeat) - rm(dummydatarepeat) - - } + # } write.csv(dummydatarepeatall, paste0("data/",repeat_table,".csv"), row.names = FALSE) cat(paste0("\n\n\n Finished generation of nested table ", h, " - ", repeat_table, "\n")) rm(dummydatarepeatall) + } -} + } NULL diff --git a/R/kobo_get_theme.R b/R/kobo_get_theme.R index 7efb515..17f1572 100644 --- a/R/kobo_get_theme.R +++ b/R/kobo_get_theme.R @@ -28,7 +28,7 @@ #' @export kobo_get_theme #' -kobo_get_theme <- function(form = "form.xls") { +kobo_get_theme <- function() { themeList <- list() diff --git a/R/kobo_left_align.R b/R/kobo_left_align.R new file mode 100644 index 0000000..37022fb --- /dev/null +++ b/R/kobo_left_align.R @@ -0,0 +1,22 @@ +#' @name kobo_left_align +#' @rdname kobo_left_align +#' @title UNHCR ggplot2 theme +#' +#' @description Left align chart title and subtitle on a ggplot2 +#' +#' @return Return better chart +#' +#' @author Edouard Legoupil - with inspiration from bbc +#' +#' @examples +#' kobo_left_align() +#' +#' @export kobo_left_align +#' + +kobo_left_align <- function(plot_name, pieces){ + grob <- ggplot2::ggplotGrob(plot_name) + n <- length(pieces) + grob$layout$l[grob$layout$name %in% pieces] <- 2 + return(grob) +} diff --git a/R/kobo_load_data.R b/R/kobo_load_data.R index f63a235..0a48be1 100644 --- a/R/kobo_load_data.R +++ b/R/kobo_load_data.R @@ -6,7 +6,7 @@ #' #' @param form The full filename of the form to be accessed (xls or xlsx file). #' It is assumed that the form is stored in the data folder. -#' +#' #' @param app The place where the function has been executed, the default is the console and the second option is the shiny app #' #' @return No return, all results will be saved inside new CSV files @@ -24,10 +24,11 @@ #' @export kobo_load_data #' -kobo_load_data <- function(form = "form.xls", app="console") { - tryCatch({ +kobo_load_data <- function(form = "form.xls", app = "console") { + tryCatch ( { ## Load all required packages############################################# - if(app=="shiny"){ + + if (app == "shiny") { progress <- shiny::Progress$new() progress$set(message = "Load Data in progress...", value = 0) on.exit(progress$close()) @@ -40,56 +41,90 @@ kobo_load_data <- function(form = "form.xls", app="console") { } updateProgress() } + + kobo_load_packages() + + ## getting project configuration variables + cat("\n\n\n Getting project configuration variables \n\n\n\n") configInfoOrigin <- kobo_get_config(form) configInfoOrigin <- configInfoOrigin[!is.na(configInfoOrigin$name),] - mainDir <- kobo_getMainDirectory() - + + + cat("\n\n\n Generate dictionnary from the xlsform \n\n\n\n") + mainDir <- kobo_getMainDirectory() kobo_dico(form) dico <- read.csv(paste0(mainDir,"/data/dico_",form,".csv"), encoding = "UTF-8", na.strings = "") - + ## Load data ####################################################################### cat("\n\n\n Load original dataset \n\n\n\n") - originalData <- read.csv(configInfoOrigin[configInfoOrigin$name=="MainDataFrame", "path"], sep = ",", encoding = "UTF-8", na.strings = "") - + + originalData <- read.csv(configInfoOrigin[configInfoOrigin$name == "MainDataFrame", "path"], sep = ",", encoding = "UTF-8", na.strings = "") + + if (ncol(originalData) == 1) { + cat("seems like you file use ; rather , variable separator.... \n") + originalData <- read.csv(configInfoOrigin[configInfoOrigin$name == "MainDataFrame", "path"], sep = ";", encoding = "UTF-8", na.strings = "") + } + ## Check to split select_multiple if data is extracted from ODK ################### - cat("\n\n\n Now split select_multiple variables \n\n\n\n") - if(app=="shiny"){ + if (app == "shiny") { progress$set(message = "Splitting Main Data File in progress...") updateProgress() } - MainDataFrame_edited <- kobo_split_multiple(originalData, dico) - + + cat("\n\n\n Now split select_multiple variables \n\n\n\n") + MainDataFrame <- kobo_split_multiple(originalData, dico) + + ## Clean variable if any ########################################################## - cat("\n\n\n Clean variable if any \n\n\n\n") - if(app=="shiny"){ + if (app == "shiny") { progress$set(message = "Cleaning Main Data File in progress...") updateProgress() } - MainDataFrame_edited <- kobo_clean(MainDataFrame_edited, dico) + + + cat("\n\n\n Clean variable if any \n\n\n\n") + MainDataFrame <- kobo_clean(MainDataFrame, dico) + + ## Join with Weight file ######################################### - cat("\n\n\n Adding weight and removing some forms \n\n\n\n") - if(nrow(configInfoOrigin)==0){ - cat("\n\n\n You need to enter the sampling methods and all required parameters in settings sheet before processed \n\n\n\n") + cat("\n\n\n Set up sampling \n\n\n\n") + + if (nrow(configInfoOrigin) == 0) { + cat("\n\n\n You need to enter the sampling methods and all required parameters in settings sheet before processed \n\n No sampling(type 1) \n\n Cluster sample (type 2) \n\n Stratified sample (type 3) \n\n") return(FALSE) } - if(length(configInfoOrigin[configInfoOrigin$name=="sample_type", "value"])!=0){ - if(configInfoOrigin[configInfoOrigin$name=="sample_type", "value"] != "No sampling(type 1)"){ - if(app=="shiny"){ + + if (length(configInfoOrigin[configInfoOrigin$name == "sample_type", "value"]) != 0) { + + if (configInfoOrigin[configInfoOrigin$name == "sample_type", "value"] != "No sampling (type 1)") { + + if (app == "shiny") { progress$set(message = "Adding weights with Main Data File in progress...") updateProgress() } - path <- configInfoOrigin[configInfoOrigin$name=="weights_info", "path"] + + path <- configInfoOrigin[configInfoOrigin$name == "weights_info", "path"] weight <- read.csv(path,stringsAsFactors = F) - variableName <- configInfoOrigin[configInfoOrigin$name=="variable_name", "value"] - MainDataFrame_edited <- left_join(x = MainDataFrame_edited, y = weight, by = variableName) + + + variableName <- configInfoOrigin[configInfoOrigin$name == "variable_name", "value"] + MainDataFrame <- left_join(x = MainDataFrame, y = weight, by = variableName) + + } } + + ## Cheking the labels matching... ################################################# - ## MainDataFrame_edited is the default root data componnents to be used -- in order to deal with nested dataset - if(app=="shiny"){ + + + ## MainDataFrame is the default root data componnents to be used -- in order to deal with nested dataset + if (app == "shiny") { + + progress$set(message = "labeling variables for Main Data File in progress...") updateProgress() } @@ -97,144 +132,198 @@ kobo_load_data <- function(form = "form.xls", app="console") { MainDataFrame_edited <- kobo_label(MainDataFrame_edited, dico) ## Save preliminary version before encoding or adding indicators ################## - cat("\n\nWrite backup before encoding or indicators calculation..\n") - write.csv(MainDataFrame_edited,paste(mainDir,"/data/MainDataFrame_edited.csv",sep = ""), row.names = FALSE, na = "") - + + + cat("\n\n Write backup before encoding or indicators calculation..\n") + write.csv(MainDataFrame,paste(mainDir,"/data/MainDataFrame-edited.csv",sep = ""), row.names = FALSE, na = "") + + + ## load all required data files ######################################### - cat("\n\nload all required data files..\n") - if(app=="shiny"){ + cat("\n\nload all required nested data files..\n") + if (app == "shiny") { progress$set(message = "loading all required data files in progress...") updateProgress() } - + + configInfo <- configInfoOrigin[startsWith(tolower(configInfoOrigin$name), "instanceid"),] + levelsOfDF <- kobo_get_dataframes_levels(form) - levelsOfDF <- levelsOfDF[levelsOfDF$name!="MainDataFrame",] - if(nrow(levelsOfDF)!=0){ - levelsOfDF[levelsOfDF$parent=="MainDataFrame","parent"] <- "MainDataFrame_edited" - } - #ataBeginRepeat <- kobo_get_begin_repeat("form2.xls") + + + levelsOfDF <- levelsOfDF[levelsOfDF$name != "MainDataFrame",] + if (nrow(levelsOfDF) != 0) { + # levelsOfDF[levelsOfDF$parent == "MainDataFrame","parent"] <- "MainDataFrame" + #} + #dataBeginRepeat <- kobo_get_begin_repeat("form2.xls") + + #dataBeginRepeat <- dataBeginRepeat$names - - if(nrow(levelsOfDF)!=0){ - + + for (dbr in levelsOfDF$name) { - cat("\n\nload all required data files..\n") - if(app=="shiny"){ + + if (app == "shiny") { progress$set(message = paste("loading",dbr,"file in progress...")) updateProgress() } - - dataFrame <- read.csv(configInfoOrigin[configInfoOrigin$name==dbr,"path"], stringsAsFactors = F) - - if(app=="shiny"){ + # dbr <- levelsOfDF$name[1] + cat("\n\nloading",dbr,"file ..\n") + dataFrame <- read.csv(configInfoOrigin[configInfoOrigin$name == dbr,"path"], stringsAsFactors = F) + + if (app == "shiny") { progress$set(message = paste("Splitting",dbr,"file in progress...")) updateProgress() } + cat(paste("Splitting",dbr,"file in progress...\n")) dataFrame <- kobo_split_multiple(dataFrame, dico) - if(app=="shiny"){ + + + if (app == "shiny") { progress$set(message = paste("Cleaning",dbr,"file in progress...")) updateProgress() } + cat(paste("Cleaning",dbr,"file in progress...\n")) dataFrame <- kobo_clean(dataFrame, dico) - if(app=="shiny"){ + + + if (app == "shiny") { progress$set(message = paste("Labeling",dbr,"file in progress...")) updateProgress() } + cat(paste("Labeling",dbr,"file in progress...\n")) dataFrame <- kobo_label(dataFrame, dico) - - write.csv(dataFrame,paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""), row.names = FALSE, na = "") - cat("\n\nload",dbr,"and create all needed files for it..\n") - - } - for (dbr in levelsOfDF$name) { - if(app=="shiny"){ - progress$set(message = paste("loading",dbr,"file in progress...")) - updateProgress() - } - dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""),stringsAsFactors = F) - child <- levelsOfDF[levelsOfDF$name==dbr, "name"] - parent <- levelsOfDF[levelsOfDF$name==dbr, "parent"] + + + cat("\n\n Saving ",dbr,"file as -edited..\n") + write.csv(dataFrame,paste(mainDir,"/data/",dbr,"-edited.csv",sep = ""), row.names = FALSE, na = "") + + # } + # + # + # cat("\n\n Join hierarchical structured if defined..\n") + # for (dbr in levelsOfDF$name) { + + # if (app == "shiny") { + # progress$set(message = paste("loading",dbr,"file in progress...")) + # updateProgress() + # } + # + # dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"-edited.csv",sep = ""),stringsAsFactors = F) + child <- levelsOfDF[levelsOfDF$name == dbr, "name"] + parent <- levelsOfDF[levelsOfDF$name == dbr, "parent"] + + cat("\n\n Join hierarchical structure between ", child, " and ", parent, " in order to calculate indicators...\n") + while (T) { - instanceIDChild <- configInfo[tolower(configInfo$name)==tolower(paste0("instanceid_",child,"_",ifelse(parent=="MainDataFrame_edited","MainDataFrame",parent))), "value"] - instanceIDParent <- configInfo[tolower(configInfo$name)==tolower(paste0("instanceid_",ifelse(parent=="MainDataFrame_edited","MainDataFrame",parent),"_",child)), "value"] - if(parent=="MainDataFrame_edited"){ - parentDf <- read.csv(paste(mainDir,"/data/",parent,".csv",sep = ""),stringsAsFactors = F) + instanceIDChild <- configInfo[tolower(configInfo$name) == tolower(paste0("instanceid_",child,"_",ifelse(parent == "household","MainDataFrame",parent))), "value"] + instanceIDParent <- configInfo[tolower(configInfo$name) == tolower(paste0("instanceid_",ifelse(parent == "household","MainDataFrame",parent),"_",child)), "value"] + + ## Case MainDataFrame called household + if (parent %in% c("household", "MainDataFrame")) { + parentDf <- read.csv(paste(mainDir,"/data/",parent,"-edited.csv",sep = ""),stringsAsFactors = F) + }else{ parentDf <- read.csv(paste(mainDir,"/data/",parent,"_edited.csv",sep = ""),stringsAsFactors = F) } + + + ## Preparing the 2 data frame for a left join - create a common key betwee 2 frames for the left_join unColChild <- dataFrame[,instanceIDChild] - dataFrame <- dataFrame[colnames(dataFrame)!=instanceIDChild] + + ## Removing this from child + dataFrame <- dataFrame[ colnames(dataFrame) != instanceIDChild] + + ## get all variables from child that are not in parent + unCN <- colnames(dataFrame)[!colnames(dataFrame) %in% colnames(parentDf)] - - if(instanceIDChild != instanceIDParent){ + + if (instanceIDChild != instanceIDParent) { unCN <- c(instanceIDChild, unCN, "jointemp") dataFrame[instanceIDChild] <- unColChild - dataFrame["jointemp"] <- unColChild - }else{ + dataFrame[ , "jointemp"] <- unColChild + } else { unCN <- c(unCN, "jointemp") - dataFrame["jointemp"] <- unColChild + dataFrame[ , "jointemp"] <- unColChild } - - parentDf["jointemp"] <- parentDf[,instanceIDParent] - + + parentDf[, "jointemp"] <- parentDf[,instanceIDParent] + dataFrame <- dataFrame[ unCN ] - - dataFrame <- left_join(dataFrame, parentDf, by="jointemp") + + ### Now ready for a left join + dataFrame <- plyr::join(dataFrame, parentDf, by = "jointemp", type = "left") dataFrame["jointemp"] <- NULL - - if(parent=="MainDataFrame_edited"){ + + + if (parent == "MainDataFrame") { + break - }else{ - child <- levelsOfDF[levelsOfDF$name==parent, "name"] - parent <- levelsOfDF[levelsOfDF$name==parent, "parent"] + } else { + child <- levelsOfDF[levelsOfDF$name == parent, "name"] + parent <- levelsOfDF[levelsOfDF$name == parent, "parent"] } } - - write.csv(dataFrame,paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""), row.names = FALSE, na = "") + + cat("\n\n Saving edited version of ", dbr, " ...\n") + write.csv(dataFrame,paste(mainDir,"/data/",dbr,"-edited.csv",sep = ""), row.names = FALSE, na = "") + } - + } + + + ## Compute indicators if defined ################################################## - cat("\n\nCompute indicators if defined..\n") - if(app=="shiny"){ + if (app == "shiny") { progress$set(message = "Computing indicators (if defined) in progress...") updateProgress() } + cat("\n\n Now computing all calculated indicators if defined..\n") result <- kobo_create_indicators(form) - if(class(result) == "try-error"){ + + if (class(result) == "try-error") { return(structure(result, class = "try-error")) } - - - + + + dico <- read.csv(paste0(mainDir,"/data/dico_",form,".csv"), encoding = "UTF-8", na.strings = "") - MainDataFrame_edited <- read.csv(paste(mainDir,"/data/MainDataFrame_edited.csv",sep = ""), encoding = "UTF-8", na.strings = "NA") + + MainDataFrame <- read.csv(paste(mainDir,"/data/MainDataFrame-edited.csv",sep = ""), encoding = "UTF-8", na.strings = "NA") + ## Re-encoding data now based on the dictionnary -- ############################## ## the xlsform dictionnary can be adjusted this script re-runned till satisfaction cat("\n\n\n Now re-encode data \n\n\n\n") cat("\n\nCompute indicators if defined..\n") - if(app=="shiny"){ + if (app == "shiny") { progress$set(message = "Re-encoding data now based on the dictionnary in progress...") updateProgress() } - - MainDataFrame_edited <- kobo_encode(MainDataFrame_edited, dico) + + + MainDataFrame <- kobo_encode(MainDataFrame, dico) + + ## loading nested frame for (dbr in levelsOfDF$name) { - dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""),stringsAsFactors = F) + dataFrame <- read.csv(paste(mainDir,"/data/",dbr,"-edited.csv",sep = ""),stringsAsFactors = F) dataFrame <- kobo_encode(dataFrame, dico) - write.csv(dataFrame,paste(mainDir,"/data/",dbr,"_edited.csv",sep = ""), row.names = FALSE, na = "") + write.csv(dataFrame,paste(mainDir,"/data/",dbr,"-encoded.csv",sep = ""), row.names = FALSE, na = "") + cat("\n\nRe-encode",dbr,"..\n") } - if(app=="shiny"){ + if (app == "shiny") { updateProgress() } - write.csv(MainDataFrame_edited,paste(mainDir,"/data/MainDataFrame_edited.csv",sep = ""), row.names = FALSE, na = "") + + write.csv(MainDataFrame,paste(mainDir,"/data/MainDataFrame-encoded.csv",sep = ""), row.names = FALSE, na = "") + return(TRUE) }, error = function(err) { - print("kobo_load_data_ERROR") + print("There was an error in the data processing step!!! \n\n") return(structure(err, class = "try-error")) }) } diff --git a/R/kobo_prepare_form.R b/R/kobo_prepare_form.R index 22d10f4..e68cd19 100644 --- a/R/kobo_prepare_form.R +++ b/R/kobo_prepare_form.R @@ -45,7 +45,7 @@ kobo_prepare_form <- function(form = "form.xls") { form_tmp <- paste(mainDir, "data", form, sep = "/", collapse = "/") - ### First review all questions from survey sheet ################################################# + # Survey sheet ###################################### survey <- tryCatch({ as.data.frame(read_excel(form_tmp, sheet = "survey"), stringsAsFactors = FALSE) #read survey sheet from the form @@ -85,6 +85,12 @@ kobo_prepare_form <- function(form = "form.xls") { ) }) + + cat("################################# \n") + cat("### Checking now survey sheet ## \n") + cat("################################# \n") + + namesOfSur <- c("type", "name" , "label") ## Rename the variable label @@ -122,11 +128,11 @@ kobo_prepare_form <- function(form = "form.xls") { } cat("Checking now for additional information within your xlsform. Note that you can insert them in the xls and re-run the function! \n \n ") - ### add column if not present ################################################# + ### Add column if not present if ("labelReport" %in% colnames(survey)) { cat(" Good: You have a column `labelReport` in your survey worksheet.\n"); } else { - cat(" No column `labelReport` in your survey worksheet. Creating a dummy one for the moment (see readme file). ...\n"); + cat(" No column `labelReport` in your survey worksheet. Creating a dummy one for the moment based on the initial one - trimmed to 80 characters (see readme file). ...\n"); survey["labelReport"] <- substr(survey[,"label"],1,80) } namesOfSur <- c(namesOfSur,"labelReport") @@ -254,7 +260,7 @@ kobo_prepare_form <- function(form = "form.xls") { survey[is.na(survey)] <- "" - #################################Styling part for survey sheet########################## + #### Styling part for survey sheet sheetname <- "survey" if (!is.null(xlsx::getSheets(wb)[[sheetname]])) xlsx::removeSheet(wb, sheetname) @@ -341,7 +347,7 @@ kobo_prepare_form <- function(form = "form.xls") { cat("\n********************Survey sheet, ready to be used*********************\n \n") - #################################### choices sheet ###################################### + # Choices sheet ###################################### choices <- tryCatch({ as.data.frame(read_excel(form_tmp, sheet = "choices"), stringsAsFactors = FALSE) #read survey sheet from the form @@ -356,9 +362,9 @@ kobo_prepare_form <- function(form = "form.xls") { check.names = F ) }) - - cat("\n \n Checking now choices sheet \n \n") - + cat("################################# \n") + cat("### Checking now choices sheet ## \n") + cat("################################# \n") ## Rename the variable label names(choices)[tolower(names(choices)) == "label::english"] <- "label" @@ -368,21 +374,42 @@ kobo_prepare_form <- function(form = "form.xls") { return(structure('Please make sure the choices sheet have the following columns "type", "name" , "label"', class = "try-error")) } - ### add column if not present ################################################# + ### add column if not present if ("order" %in% colnames(choices)) { - cat("1- Good: You have a column `order` in your choices worksheet.\n"); + cat(" Good: You have a column `order` in your choices worksheet.\n"); } else { - cat("1- No column `order` in your choices worksheet. Creating a dummy one for the moment...\n"); + cat(" No column `order` in your choices worksheet. Creating a dummy one for the moment...\n"); choices$order <- "" } if ("labelReport" %in% colnames(choices)) { - cat("2- Good: You have a column `labelReport` in your choices worksheet.\n"); - } else { - cat("2- No column `labelReport` in your choices worksheet. Creating a dummy one for the moment...\n"); - choices["labelReport"] <- substr(choices[,"label"],1,80) - } - - namesOfCho <- c("list_name", "name", "label", "labelReport", "order") + cat(" Good: You have a column `labelReport` in your choices worksheet.\n"); + } else { + cat(" No column `labelReport` in your choices worksheet. Creating a dummy one for the moment based on the initial one - trimmed to 50 characters...\n"); + choices["labelReport"] <- substr(choices[,"label"],1,50) + } + + if ("weight" %in% colnames(choices)) + { + cat(" Good: You have a column `weight` in your `choices` worksheet.\n"); + } else + {cat(" No column `weight` in your `choices` worksheet. Creating a dummy one for the moment...\n"); + choices$weight <- ""} + + if ("recategorise" %in% colnames(choices)) + { + cat(" Good: You have a column `recategorise` in your `choices` worksheet.\n"); + } else + {cat(" No column `recategorise` in your `choices` worksheet. Creating a dummy one for the moment...\n"); + choices$recategorise <- ""} + + if ("score" %in% colnames(choices)) + { + cat(" Good: You have a column `score` in your `choices` worksheet.\n"); + } else + {cat(" No column `score` in your `choices` worksheet. Creating a dummy one for the moment...\n"); + choices$score <- ""} + + namesOfCho <- c("list_name", "name", "label", "labelReport", "order", "weight","score","recategorise") choices <- choices[ ,namesOfCho] sheetname <- "choices" @@ -408,205 +435,64 @@ kobo_prepare_form <- function(form = "form.xls") { xlsx::setColumnWidth(choicesSheet, 2:3, 30) cat("\n********************Choices sheet, ready to be used*********************\n \n") - #################################### indicator sheet ###################################### - cat("\n \n Checking now indicator sheet \n \n") - indicator <- tryCatch({ - as.data.frame(read_excel(form_tmp, sheet = "indicator"),stringsAsFactors = FALSE) - }, error = function(err) { - data.frame( - type = character(), - fullname = character(), - label = character(), - chapter = character(), - disaggregation = character(), - correlate = character(), - sensitive = character(), - anonymise = character(), - cluster = character(), - predict = character(), - variable = character(), - mappoint = character(), - mappoly = character(), - structuralequation = character(), - frame = character(), - listname = character(), - calculation = character(), - stringsAsFactors = FALSE - ) - }) - if ("type" %in% colnames(indicator)) { - cat("1- Good: You have a column `type` in your indicator worksheet.\n"); - } else { - cat("1- No column `type` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$type <- "" - } - if ("fullname" %in% colnames(indicator)) { - cat("2- Good: You have a column `fullname` in your indicator worksheet.\n"); - } else { - cat("2- No column `fullname` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$fullname <- "" - } - if ("label" %in% colnames(indicator)) { - cat("3- Good: You have a column `label` in your indicator worksheet.\n"); - } else { - cat("3- No column `label` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$label <- "" - } - if ("chapter" %in% colnames(indicator)) { - cat("4- Good: You have a column `chapter` in your indicator worksheet.\n"); - } else { - cat("4- No column `chapter` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$chapter <- "" - } - if ("disaggregation" %in% colnames(indicator)) { - cat("5- Good: You have a column `disaggregation` in your indicator worksheet.\n"); - } else { - cat("5- No column `disaggregation` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$disaggregation <- "" - } - if ("correlate" %in% colnames(indicator)) { - cat("6- Good: You have a column `correlate` in your indicator worksheet.\n"); - } else { - cat("6- No column `correlate` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$correlate <- "" - } + ### Settings sheet ###################################### - if ("sensitive" %in% colnames(indicator)) { - cat("7- Good: You have a column `sensitive` in your indicator worksheet.\n"); - } else { - cat("7- No column `sensitive` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$sensitive <- "" - } - if ("anonymise" %in% colnames(indicator)) { - cat("8- Good: You have a column `anonymise` in your indicator worksheet.\n"); - } else { - cat("8- No column `anonymise` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$anonymise <- "" - } - if ("cluster" %in% colnames(indicator)) { - cat("9- Good: You have a column `cluster` in your indicator worksheet.\n"); - } else { - cat("9- No column `cluster` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$cluster <- "" - } - if ("predict" %in% colnames(indicator)) { - cat("10- Good: You have a column `predict` in your indicator worksheet.\n"); - } else { - cat("10- No column `predict` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$predict <- "" - } - if ("variable" %in% colnames(indicator)) { - cat("11- Good: You have a column `variable` in your indicator worksheet.\n"); - } else { - cat("11- No column `variable` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$variable <- "" - } - if ("mappoint" %in% colnames(indicator)) { - cat("12- Good: You have a column `mappoint` in your indicator worksheet.\n"); - } else { - cat("12- No column `mappoint` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$mappoint <- "" - } - if ("mappoly" %in% colnames(indicator)) { - cat("13- Good: You have a column `mappoly` in your indicator worksheet.\n"); - } else { - cat("13- No column `mappoly` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$mappoly <- "" - } - if ("structuralequation" %in% colnames(indicator)) { - cat("14- Good: You have a column `structuralequation` in your indicator worksheet.\n"); - } else { - cat("14- No column `structuralequation` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$structuralequation <- "" - } - if ("frame" %in% colnames(indicator)) { - cat("15- Good: You have a column `frame` in your indicator worksheet.\n"); - } else { - cat("15- No column `frame` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$frame <- "" - } - if ("listname" %in% colnames(indicator)) { - cat("16- Good: You have a column `listname` in your indicator worksheet.\n"); - } else { - cat("16- No column `listname` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$listname <- "" - } - if ("calculation" %in% colnames(indicator)) { - cat("17- Good: You have a column `calculation` in your indicator worksheet.\n"); - } else { - cat("17- No column `calculation` in your indicator worksheet. Creating a dummy one for the moment...\n"); - indicator$calculation <- "" - } - indicator <- indicator[ ,c("type","fullname","label", "chapter", "disaggregation", "correlate", "sensitive", - "anonymise", "cluster", "predict", "variable", "mappoint", "mappoly", "structuralequation", "frame", "listname","calculation")] + cat("\n\n################################### \n") + cat("### Checking now settings sheet ## \n") + cat("################################### \n\n") - sheetname <- "indicator" - if (!is.null(xlsx::getSheets(wb)[[sheetname]])) - xlsx::removeSheet(wb, sheetname) - indicatorSheet <- xlsx::createSheet(wb, sheetName = sheetname) - xlsx::addDataFrame(indicator, indicatorSheet, col.names = TRUE, row.names = FALSE) - from <- "A1" - to <- dfref[dfref$key == length(indicator),"val"] - xlsx::addAutoFilter(indicatorSheet, paste(from,":",to,sep = "")) - rows <- xlsx::getRows(indicatorSheet) # get rows - cells <- xlsx::getCells(rows) - headerSt <- xlsx::CellStyle(wb) + - xlsx::Font(wb, isBold = TRUE, isItalic = FALSE, color = "white", heightInPoints = 13) + - xlsx::Fill(backgroundColor = "GREY_50_PERCENT",foregroundColor = "GREY_50_PERCENT", - pattern = "SOLID_FOREGROUND") + - xlsx::Border(color = "GREY_80_PERCENT", position = c("TOP", "BOTTOM"), "BORDER_THIN") - highlight <- paste("1",c(1:length(indicator)),sep = ".") - lapply(names(cells[highlight]), - function(ii) xlsx::setCellStyle(cells[[ii]], headerSt)) - xlsx::autoSizeColumn(indicatorSheet, 1:length(survey)) - cat("\n********************Indicator sheet, ready to be used*********************\n \n") - #################################### settings sheet ###################################### - cat("\n \n Checking now settings sheet \n \n") - settings <- tryCatch({ - as.data.frame(read_excel(form_tmp, sheet = "settings"), - stringsAsFactors = FALSE) - }, error = function(err) { - data.frame( - form_title = character(), - form_id = character(), - default_language = character(), - stringsAsFactors = FALSE - ) - }) + settings <- tryCatch({ + as.data.frame(read_excel(form_tmp, sheet = "settings"), + stringsAsFactors = FALSE) + }, error = function(err) { + data.frame( + form_title = character(), + form_id = character(), + default_language = character(), + stringsAsFactors = FALSE + ) + }) + + sheetname <- "settings" + if (!is.null(xlsx::getSheets(wb)[[sheetname]])) + xlsx::removeSheet(wb, sheetname) + settingsSheet <- xlsx::createSheet(wb, sheetName = sheetname) #create sheet with settings name + xlsx::addDataFrame(settings, settingsSheet, col.names = TRUE, row.names = FALSE) #add settings data frame to this sheet + + from <- "A1" + to <- dfref[dfref$key == length(settings),"val"] + + xlsx::addAutoFilter(settingsSheet, paste(from,":",to,sep = "")) + rows <- xlsx::getRows(settingsSheet) # get rows + cells <- xlsx::getCells(rows) + headerSt <- xlsx::CellStyle(wb) + + xlsx::Font(wb, isBold = TRUE, isItalic = FALSE, color = "white", heightInPoints = 13) + + xlsx::Fill(backgroundColor = "GREY_50_PERCENT",foregroundColor = "GREY_50_PERCENT", + pattern = "SOLID_FOREGROUND") + + xlsx::Border(color = "GREY_80_PERCENT", position = c("TOP", "BOTTOM"), "BORDER_THIN") + highlight <- paste("1",c(1:length(settings)),sep = ".") + lapply(names(cells[highlight]), + function(ii) xlsx::setCellStyle(cells[[ii]], headerSt)) + xlsx::autoSizeColumn(settingsSheet, 1:length(survey)) + cat("\n******************** Settings sheet, ready to be used*********************\n \n") + + + + + + + + ### Analysis settings sheet ###################################### + + + cat("\n\n######################################### \n") + cat("### Checking now analysis settings sheet ## \n") + cat("############################################ \n\n") - sheetname <- "settings" - if (!is.null(xlsx::getSheets(wb)[[sheetname]])) - xlsx::removeSheet(wb, sheetname) - settingsSheet <- xlsx::createSheet(wb, sheetName = sheetname) #create sheet with settings name - xlsx::addDataFrame(settings, settingsSheet, col.names = TRUE, row.names = FALSE) #add settings data frame to this sheet - from <- "A1" - to <- dfref[dfref$key == length(settings),"val"] - xlsx::addAutoFilter(settingsSheet, paste(from,":",to,sep = "")) - rows <- xlsx::getRows(settingsSheet) # get rows - cells <- xlsx::getCells(rows) - headerSt <- xlsx::CellStyle(wb) + - xlsx::Font(wb, isBold = TRUE, isItalic = FALSE, color = "white", heightInPoints = 13) + - xlsx::Fill(backgroundColor = "GREY_50_PERCENT",foregroundColor = "GREY_50_PERCENT", - pattern = "SOLID_FOREGROUND") + - xlsx::Border(color = "GREY_80_PERCENT", position = c("TOP", "BOTTOM"), "BORDER_THIN") - highlight <- paste("1",c(1:length(settings)),sep = ".") - lapply(names(cells[highlight]), - function(ii) xlsx::setCellStyle(cells[[ii]], headerSt)) - xlsx::autoSizeColumn(settingsSheet, 1:length(survey)) - cat("\n********************Settings sheet, ready to be used*********************\n \n") - - - - - - - - #################################### analysis settings sheet ###################################### - cat("\n \n Checking now analysis settings sheet \n \n") analysisSettings <- tryCatch({ as.data.frame(read_excel(form_tmp, sheet = "analysisSettings"), stringsAsFactors = FALSE) @@ -620,131 +506,131 @@ kobo_prepare_form <- function(form = "form.xls") { stringsAsFactors = FALSE ) }) - - if(!"sample_type" %in% analysisSettings$name){ + + if (!"sample_type" %in% analysisSettings$name) { analysisSettings <- rbind(analysisSettings, data.frame(name = "sample_type", label = "Sample type of the project", options = "1. No sampling(type 1) 2. Cluster sample (type 2) 3. Stratified sample (type 3)", - value = NA, - path =NA, + value = "No sampling (type 1)", + path = NA, stringsAsFactors = FALSE) ) } - - if(!"variable_name" %in% analysisSettings$name){ + + if (!"variable_name" %in% analysisSettings$name) { analysisSettings <- rbind(analysisSettings, data.frame(name = "variable_name", label = "If there is sampling, select the name of cluster variable that will be used to join the weight file with the main file, please make sure the name of this variable exists in both files", options = NA, value = NA, - path =NA, + path = NA, stringsAsFactors = FALSE) ) } - - if(!"weights_info" %in% analysisSettings$name){ + + if (!"weights_info" %in% analysisSettings$name) { analysisSettings <- rbind(analysisSettings, data.frame(name = "weights_info", label = "If there is sampling, weights file that will be used in Stratified or cluster sample", options = NA, value = NA, - path =NA, + path = NA, stringsAsFactors = FALSE) ) } - - if(!"weightsVariable" %in% analysisSettings$name){ + + if (!"weightsVariable" %in% analysisSettings$name) { analysisSettings <- rbind(analysisSettings, data.frame(name = "weightsVariable", label = "If there is sampling, the variable that contains the weights in weights file", options = NA, value = NA, - path =NA, + path = NA, stringsAsFactors = FALSE) ) } - - if(!"numberOfClusters" %in% analysisSettings$name){ + + if (!"numberOfClusters" %in% analysisSettings$name) { analysisSettings <- rbind(analysisSettings, data.frame(name = "numberOfClusters", label = "If the sample type is cluster sample, enter number of clusters", options = NA, value = NA, - path =NA, + path = NA, stringsAsFactors = FALSE) ) } - - if(!"cleaning_log" %in% analysisSettings$name){ + + if (!"cleaning_log" %in% analysisSettings$name) { analysisSettings <- rbind(analysisSettings, data.frame(name = "cleaning_log", label = "cleaning log plan for the project", options = "1. Yes 2. No, 3. csv filename", - value = NA, - path =NA, + value = "2. No", + path = NA, stringsAsFactors = FALSE) ) } - - if(!"MainDataFrame" %in% analysisSettings$name){ + + if (!"MainDataFrame" %in% analysisSettings$name) { analysisSettings <- rbind(analysisSettings, data.frame(name = "MainDataFrame", label = "Name and the path of MainDataFrame", options = NA, - value = NA, - path =NA, + value = "MainDataFrame", + path = paste0(mainDir,"/data/MainDataFrame.csv"), stringsAsFactors = FALSE) ) } - + levelsOfDF <- kobo_get_dataframes_levels(form) - levelsOfDF <- levelsOfDF[levelsOfDF$name!="MainDataFrame",] - - if(nrow(levelsOfDF)!=0){ + levelsOfDF <- levelsOfDF[levelsOfDF$name != "MainDataFrame",] + + if (nrow(levelsOfDF) != 0) { for (dbr in levelsOfDF$name) { - child <- levelsOfDF[levelsOfDF$name==dbr, "name"] - parent <- levelsOfDF[levelsOfDF$name==dbr, "parent"] - - if(!dbr %in% analysisSettings$name){ + child <- levelsOfDF[levelsOfDF$name == dbr, "name"] + parent <- levelsOfDF[levelsOfDF$name == dbr, "parent"] + + if (!dbr %in% analysisSettings$name) { analysisSettings <- rbind(analysisSettings, data.frame(name = dbr, label = paste("Name and the path of", dbr), options = NA, - value = NA, - path =NA, + value = paste0( dbr,".csv"), + path = paste0(mainDir,"/data/", dbr,".csv"), stringsAsFactors = FALSE) ) } - - if(!paste0("instanceID_", child, "_", parent) %in% analysisSettings$name){ + + if (!paste0("instanceID_", child, "_", parent) %in% analysisSettings$name) { analysisSettings <- rbind(analysisSettings, data.frame(name = paste0("instanceID_", child, "_", parent), label = paste0("The instanceID between the child (", child, ") and the parent (", parent, ")" ), options = NA, - value = NA, - path =NA, + value = "instanceID", + path = NA, stringsAsFactors = FALSE) ) } - - if(!paste0("instanceID_", parent, "_", child) %in% analysisSettings$name){ + + if (!paste0("instanceID_", parent, "_", child) %in% analysisSettings$name) { analysisSettings <- rbind(analysisSettings, data.frame(name = paste0("instanceID_", parent, "_", child), label = paste0("The instanceID between the parent (", parent, ") and the child (", child, ")" ), options = NA, - value = NA, - path =NA, + value = "instanceID", + path = NA, stringsAsFactors = FALSE) ) } - - + + } } - - - + + + sheetname <- "analysisSettings" if (!is.null(xlsx::getSheets(wb)[[sheetname]])) xlsx::removeSheet(wb, sheetname) @@ -764,13 +650,207 @@ kobo_prepare_form <- function(form = "form.xls") { lapply(names(cells[highlight]), function(ii) xlsx::setCellStyle(cells[[ii]], headerSt)) xlsx::autoSizeColumn(settingsSheet, 1:length(survey)) - cat("\n********************analysis Settings sheet, ready to be used*********************\n \n") - - + cat("\n******************** Project Analysis Settings sheet, ready to be used*********************\n \n") + + + + + ## Indicator sheet ###################################### + + + cat("################################### \n") + cat("### Checking now indicator sheet ## \n") + cat("################################### \n") + + + indicator <- tryCatch({ + as.data.frame(read_excel(form_tmp, sheet = "indicator"),stringsAsFactors = FALSE) + }, error = function(err) { + data.frame( + type = character(), + fullname = character(), + labelReport = character(), + hintReport = character(), + frame = character(), + listname = character(), + calculation = character(), + chapter = character(), + variable = character(), + disaggregation = character(), + correlate = character(), + anonymise = character(), + cluster = character(), + predict = character(), + variable = character(), + mappoint = character(), + mappoly = character(), + structuralequation.risk = character(), + structuralequation.coping = character(), + structuralequation.resilience = character(), + stringsAsFactors = FALSE + ) + } + ) + if ("type" %in% colnames(indicator)) { + cat(" Good: You have a column `type` in your indicator worksheet.\n"); + } else { + cat(" No column `type` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$type <- "" + } + if ("fullname" %in% colnames(indicator)) { + cat(" Good: You have a column `fullname` in your indicator worksheet.\n"); + } else { + cat(" No column `fullname` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$fullname <- "" + } + if ("frame" %in% colnames(indicator)) { + cat(" Good: You have a column `frame` in your indicator worksheet.\n"); + } else { + cat(" No column `frame` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$frame <- "" + } + if ("labelReport" %in% colnames(indicator)) { + cat(" Good: You have a column `labelReport` in your indicator worksheet.\n"); + } else { + cat(" No column `labelReport` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$labelReport <- "" + } + if ("hintReport" %in% colnames(indicator)) { + cat(" Good: You have a column `hintReport` in your indicator worksheet.\n"); + } else { + cat(" No column `hintReport` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$hintReport <- "" + } + if ("listname" %in% colnames(indicator)) { + cat(" Good: You have a column `listname` in your indicator worksheet.\n"); + } else { + cat(" No column `listname` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$listname <- "" + } + if ("calculation" %in% colnames(indicator)) { + cat(" Good: You have a column `calculation` in your indicator worksheet.\n"); + } else { + cat(" No column `calculation` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$calculation <- "" + } + if ("chapter" %in% colnames(indicator)) { + cat(" Good: You have a column `chapter` in your indicator worksheet.\n"); + } else { + cat(" No column `chapter` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$chapter <- "" + } + if ("variable" %in% colnames(indicator)) { + cat(" Good: You have a column `variable` in your indicator worksheet.\n"); + } else { + cat(" No column `variable` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$variable <- "" + } + if ("disaggregation" %in% colnames(indicator)) { + cat(" Good: You have a column `disaggregation` in your indicator worksheet.\n"); + } else { + cat(" No column `disaggregation` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$disaggregation <- "" + } + if ("correlate" %in% colnames(indicator)) { + cat(" Good: You have a column `correlate` in your indicator worksheet.\n"); + } else { + cat(" No column `correlate` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$correlate <- "" + } + if ("anonymise" %in% colnames(indicator)) { + cat(" Good: You have a column `anonymise` in your indicator worksheet.\n"); + } else { + cat(" No column `anonymise` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$anonymise <- "" + } + if ("cluster" %in% colnames(indicator)) { + cat(" Good: You have a column `cluster` in your indicator worksheet.\n"); + } else { + cat(" No column `cluster` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$cluster <- "" + } + if ("predict" %in% colnames(indicator)) { + cat(" Good: You have a column `predict` in your indicator worksheet.\n"); + } else { + cat(" No column `predict` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$predict <- "" + } + if ("variable" %in% colnames(indicator)) { + cat(" Good: You have a column `variable` in your indicator worksheet.\n"); + } else { + cat(" No column `variable` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$variable <- "" + } + if ("mappoint" %in% colnames(indicator)) { + cat(" Good: You have a column `mappoint` in your indicator worksheet.\n"); + } else { + cat(" No column `mappoint` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$mappoint <- "" + } + if ("mappoly" %in% colnames(indicator)) { + cat(" Good: You have a column `mappoly` in your indicator worksheet.\n"); + } else { + cat(" No column `mappoly` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$mappoly <- "" + } + if ("structuralequation.risk" %in% colnames(indicator)) { + cat(" Good: You have a column `structuralequation.risk` in your indicator worksheet. This will be used to configure the vulnerability structural equation model\n"); + } else { + cat(" No column `structuralequation.risk` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$structuralequation.risk <- "" + } + if ("structuralequation.coping" %in% colnames(indicator)) { + cat(" Good: You have a column `structuralequation.coping` in your indicator worksheet. This will be used to configure the vulnerability structural equation model\n"); + } else { + cat(" No column `structuralequation.coping` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$structuralequation.coping <- "" + } + if ("structuralequation.resilience" %in% colnames(indicator)) { + cat(" Good: You have a column `structuralequation.resilience` in your indicator worksheet. This will be used to configure the vulnerability structural equation model\n"); + } else { + cat(" No column `structuralequation.resilience` in your indicator worksheet. Creating a dummy one for the moment...\n"); + indicator$structuralequation.resilience <- "" + } + + indicator <- indicator[ ,c("type","fullname","labelReport", "hintReport", + "frame", "listname","calculation", + "chapter", "variable", "disaggregation", "correlate", + "anonymise", "cluster", "predict", "variable", "mappoint", "mappoly", + "structuralequation.risk","structuralequation.coping","structuralequation.resilience")] + + sheetname <- "indicator" + if (!is.null(xlsx::getSheets(wb)[[sheetname]])) + xlsx::removeSheet(wb, sheetname) + indicatorSheet <- xlsx::createSheet(wb, sheetName = sheetname) + xlsx::addDataFrame(indicator, indicatorSheet, col.names = TRUE, row.names = FALSE) + from <- "A1" + to <- dfref[dfref$key == length(indicator),"val"] + xlsx::addAutoFilter(indicatorSheet, paste(from,":",to,sep = "")) + + # get rows + rows <- xlsx::getRows(indicatorSheet) + cells <- xlsx::getCells(rows) + headerSt <- xlsx::CellStyle(wb) + + xlsx::Font(wb, isBold = TRUE, isItalic = FALSE, color = "white", heightInPoints = 13) + + xlsx::Fill(backgroundColor = "GREY_50_PERCENT",foregroundColor = "GREY_50_PERCENT", + pattern = "SOLID_FOREGROUND") + + xlsx::Border(color = "GREY_80_PERCENT", position = c("TOP", "BOTTOM"), "BORDER_THIN") + highlight <- paste("1",c(1:length(indicator)),sep = ".") + lapply(names(cells[highlight]), + function(ii) xlsx::setCellStyle(cells[[ii]], headerSt)) + xlsx::autoSizeColumn(indicatorSheet, 1:length(survey)) + cat("\n******************** Indicator sheet, ready to be used *********************\n \n") + + if (file.exists(form_tmp)) file.remove(form_tmp) xlsx::saveWorkbook(wb, form_tmp) + + + cat("\n******************** The XLSFORM has now been extended to include your analysis plan *********************\n \n") + + }, error = function(err) { - print("kobo_prepare_form_ERROR") + print("There was an error in the xlsform preparation step!!! \n\n") return(structure(err, class = "try-error")) }) diff --git a/R/kobo_unhcr_style_bar.R b/R/kobo_unhcr_style_bar.R new file mode 100644 index 0000000..9e5ae93 --- /dev/null +++ b/R/kobo_unhcr_style_bar.R @@ -0,0 +1,55 @@ +#' @name kobo_unhcr_style_bar +#' @rdname kobo_unhcr_style_bar +#' @title UNHCR ggplot2 theme +#' +#' @description Return ggplot2 styling for bar chart +#' +#' @return Return UNHCR Style +#' +#' @author Edouard Legoupil - with inspiration from bbc +#' +#' @examples +#' kobo_unhcr_style_bar() +#' +#' @export kobo_unhcr_style_bar +#' + +kobo_unhcr_style_bar <- function() { + font <- "Lato" + ggplot2::theme( + + #This sets the font, size, type and colour of text for the chart's title + plot.title = ggplot2::element_text(family = font, size = 12, face = "bold", color = "#222222"), + + #This sets the font, size, type and colour of text for the chart's subtitle, as well as setting a margin between the title and the subtitle + plot.subtitle = ggplot2::element_text(family = font, size = 11, margin = ggplot2::margin(9,0,9,0)), + plot.caption = ggplot2::element_blank(), + + #This sets the position and alignment of the legend, removes a title and backround for it and sets the requirements for any text within the legend. The legend may often need some more manual tweaking when it comes to its exact position based on the plot coordinates. + legend.position = "top", + legend.text.align = 0, + legend.background = ggplot2::element_blank(), + legend.title = ggplot2::element_blank(), + legend.key = ggplot2::element_blank(), + legend.text = ggplot2::element_text(family = font, size = 9, color = "#222222"), + + #This sets the text font, size and colour for the axis test, as well as setting the margins and removes lines and ticks. In some cases, axis lines and axis ticks are things we would want to have in the chart + axis.title = ggplot2::element_blank(), + axis.text = ggplot2::element_text(family = font, size = 11, color = "#222222"), + axis.text.x = ggplot2::element_text(margin = ggplot2::margin(5, b = 9)), + axis.ticks = ggplot2::element_blank(), + axis.line = ggplot2::element_blank(), + + #This removes all minor gridlines and adds major y gridlines. In many cases you will want to change this to remove y gridlines and add x gridlines. + panel.grid.minor = ggplot2::element_blank(), + panel.grid.major.x = ggplot2::element_line(color = "#cbcbcb"), + panel.grid.major.y = ggplot2::element_blank(), + + #This sets the panel background as blank, removing the standard grey ggplot background colour from the plot + panel.background = ggplot2::element_blank(), + + #This sets the panel background for facet-wrapped plots to white, removing the standard grey ggplot background colour and sets the title size of the facet-wrap title to font size 22 + strip.background = ggplot2::element_rect(fill = "white"), + strip.text = ggplot2::element_text(size = 11, hjust = 0) + ) + } diff --git a/R/kobo_unhcr_style_histo.R b/R/kobo_unhcr_style_histo.R new file mode 100644 index 0000000..bd4917d --- /dev/null +++ b/R/kobo_unhcr_style_histo.R @@ -0,0 +1,55 @@ +#' @name kobo_unhcr_style_histo +#' @rdname kobo_unhcr_style_histo +#' @title UNHCR ggplot2 theme +#' +#' @description Return ggplot2 styling for histogram +#' +#' @return Return UNHCR Style +#' +#' @author Edouard Legoupil - with inspiration from bbc +#' +#' @examples +#' kobo_unhcr_style_histo() +#' +#' @export kobo_unhcr_style_histo +#' + +kobo_unhcr_style_histo <- function() { + font <- "Lato" + ggplot2::theme( + + #This sets the font, size, type and colour of text for the chart's title + plot.title = ggplot2::element_text(family = font, size = 12, face = "bold", color = "#222222"), + + #This sets the font, size, type and colour of text for the chart's subtitle, as well as setting a margin between the title and the subtitle + plot.subtitle = ggplot2::element_text(family = font, size = 11, margin = ggplot2::margin(9,0,9,0)), + plot.caption = ggplot2::element_blank(), + + #This sets the position and alignment of the legend, removes a title and backround for it and sets the requirements for any text within the legend. The legend may often need some more manual tweaking when it comes to its exact position based on the plot coordinates. + legend.position = "top", + legend.text.align = 0, + legend.background = ggplot2::element_blank(), + legend.title = ggplot2::element_blank(), + legend.key = ggplot2::element_blank(), + legend.text = ggplot2::element_text(family = font, size = 9, color = "#222222"), + + #This sets the text font, size and colour for the axis test, as well as setting the margins and removes lines and ticks. In some cases, axis lines and axis ticks are things we would want to have in the chart + axis.title = ggplot2::element_blank(), + axis.text = ggplot2::element_text(family = font, size = 11, color = "#222222"), + axis.text.x = ggplot2::element_text(margin = ggplot2::margin(5, b = 9)), + axis.ticks = ggplot2::element_blank(), + axis.line = ggplot2::element_blank(), + + #This removes all minor gridlines and adds major y gridlines. In many cases you will want to change this to remove y gridlines and add x gridlines. + panel.grid.minor = ggplot2::element_blank(), + panel.grid.major.y = ggplot2::element_line(color = "#cbcbcb"), + panel.grid.major.x = ggplot2::element_blank(), + + #This sets the panel background as blank, removing the standard grey ggplot background colour from the plot + panel.background = ggplot2::element_blank(), + + #This sets the panel background for facet-wrapped plots to white, removing the standard grey ggplot background colour and sets the title size of the facet-wrap title to font size 22 + strip.background = ggplot2::element_rect(fill = "white"), + strip.text = ggplot2::element_text(size = 11, hjust = 0) + ) + } diff --git a/R/kobo_unhcr_style_map.R b/R/kobo_unhcr_style_map.R new file mode 100644 index 0000000..db994a6 --- /dev/null +++ b/R/kobo_unhcr_style_map.R @@ -0,0 +1,62 @@ +#' @name kobo_unhcr_style_map +#' @rdname kobo_unhcr_style_map +#' @title UNHCR ggplot2 theme +#' +#' @description Return ggplot2 styling for maps +#' +#' @return Return UNHCR Style +#' +#' @author Edouard Legoupil - +#' +#' @examples +#' kobo_unhcr_style_map() +#' +#' @export kobo_unhcr_style_map +#' + +kobo_unhcr_style_map <- function() { + font <- "Lato" + ggplot2::theme_minimal() + + ggplot2::theme( + #------------ + ## Plot + # plot.background = element_rect(fill = "transparent",colour = NA), + # plot.background = element_rect(fill = "#f5f5f2", color = NA), + plot.title = element_text(face = "bold", size = 12, hjust = 0, color = "#4e4d47"), + plot.subtitle = element_text(size = 8, hjust = 0, color = "#4e4d47", margin = margin(b = -0.1, t = -0.1, l = 2, unit = "cm"), debug = F), + plot.margin = unit(c(.5,.5,.2,.5), "cm"), + plot.caption = element_text(size = 6, hjust = 0.92, margin = margin(t = 0.2, b = 0, unit = "cm"), color = "#939184"), + + #------------ + ## Panel + panel.border = element_blank(), + # panel.grid.minor = element_line(color = "#ebebe5", size = 0.2), + panel.grid.major = element_line(color = "#ebebe5", size = 0.2), + panel.grid.minor = element_blank(), + # panel.background = element_rect(fill = "#f5f5f2", color = NA), + panel.spacing = unit(c(-.1,0.7,.2,1.7), "cm"), + + #------------ + ## legend + legend.title = element_text(size = 8), + legend.text = element_text(size = 7, hjust = 0, color = "#4e4d47"), + legend.position = "bottom", + legend.box = "horizontal", + # legend.position = c(0.8, 0.03), + legend.text.align = 0, + #legend.background = element_rect(fill = "transparent",colour = NA), + # legend.background = element_rect(fill = "#f5f5f2", color = NA), + legend.background = element_rect(fill = alpha('white', 0.0), color = NA), + + #------------ + ## Axis + axis.line = element_blank(), + axis.text.x = element_blank(), + axis.text.y = element_blank(), + axis.ticks = element_blank(), + axis.title.x = element_blank(), + axis.title.y = element_blank() + + + ) + } diff --git a/R/kobo_unhcr_style_scatter.R b/R/kobo_unhcr_style_scatter.R new file mode 100644 index 0000000..99a539b --- /dev/null +++ b/R/kobo_unhcr_style_scatter.R @@ -0,0 +1,55 @@ +#' @name kobo_unhcr_style_scatter +#' @rdname kobo_unhcr_style_scatter +#' @title UNHCR ggplot2 theme +#' +#' @description Return ggplot2 styling for scatter plot +#' +#' @return Return UNHCR Style +#' +#' @author Edouard Legoupil - with inspiration from bbc +#' +#' @examples +#' kobo_unhcr_style_scatter() +#' +#' @export kobo_unhcr_style_scatter +#' + +kobo_unhcr_style_scatter <- function() { + font <- "Lato" + ggplot2::theme( + + #This sets the font, size, type and colour of text for the chart's title + plot.title = ggplot2::element_text(family = font, size = 12, face = "bold", color = "#222222"), + + #This sets the font, size, type and colour of text for the chart's subtitle, as well as setting a margin between the title and the subtitle + plot.subtitle = ggplot2::element_text(family = font, size = 11, margin = ggplot2::margin(9,0,9,0)), + plot.caption = ggplot2::element_blank(), + + #This sets the position and alignment of the legend, removes a title and backround for it and sets the requirements for any text within the legend. The legend may often need some more manual tweaking when it comes to its exact position based on the plot coordinates. + legend.position = "top", + legend.text.align = 0, + legend.background = ggplot2::element_blank(), + legend.title = ggplot2::element_blank(), + legend.key = ggplot2::element_blank(), + legend.text = ggplot2::element_text(family = font, size = 9, color = "#222222"), + + #This sets the text font, size and colour for the axis test, as well as setting the margins and removes lines and ticks. In some cases, axis lines and axis ticks are things we would want to have in the chart + axis.title = ggplot2::element_blank(), + axis.text = ggplot2::element_text(family = font, size = 11, color = "#222222"), + axis.text.x = ggplot2::element_text(margin = ggplot2::margin(5, b = 9)), + axis.ticks = ggplot2::element_blank(), + axis.line = ggplot2::element_blank(), + + #This removes all minor gridlines and adds major y gridlines. In many cases you will want to change this to remove y gridlines and add x gridlines. + panel.grid.minor = ggplot2::element_blank(), + panel.grid.major.x = ggplot2::element_line(color = "#cbcbcb"), + panel.grid.major.y = ggplot2::element_line(color = "#cbcbcb"), + + #This sets the panel background as blank, removing the standard grey ggplot background colour from the plot + panel.background = ggplot2::element_blank(), + + #This sets the panel background for facet-wrapped plots to white, removing the standard grey ggplot background colour and sets the title size of the facet-wrap title to font size 22 + strip.background = ggplot2::element_rect(fill = "white"), + strip.text = ggplot2::element_text(size = 11, hjust = 0) + ) + } diff --git a/README.md b/README.md index 8591317..1271fe3 100644 --- a/README.md +++ b/README.md @@ -54,21 +54,30 @@ The `koboloadeR` package allows to: To be able to use koboloadeR you will need: - * Java JRE (https://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html): JAVA is required to manipulate excel files. + +## Software installation - * R (https://cran.rstudio.com/). For Windows, choose "install R for the first time". + 1. Install either [Java JRE](https://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html): JAVA is required to manipulate excel files. - * **Only for windows user** RTools (https://cran.r-project.org/bin/windows/Rtools/) - This executable is needed to install the package from github. +If you install, the `JDK` (Java Development kit), Please make sure that `JAVA_HOME` is actually recorded as an [Environment Variable](https://java.com/en/download/help/path.xml). - * R Studio (https://www.rstudio.com/products/rstudio/download/#download) +If you install, the `JRE` (Java Runtime Environment), Please make sure that `JRE_HOME` is actually recorded as an [Environment Variable](https://confluence.atlassian.com/doc/setting-the-java_home-variable-in-windows-8895.html). - -## Software installation - * Install Java JRE: please make sure that JAVA_HOME is actually recorded as an [Environment Variable](https://java.com/en/download/help/path.xml). Note in some case, you may need to reboot your computer to ensure that this environement variable is properly accounted for. - * Install R: follow instruction from the installer. - * Install RTools: follow instruction from the installer. - * Install R Studio: follow instruction from the installer - * Launch R Studio +Once in R, you may double-check that Environement variable are correctly, i.e. JAVA_HOME or JRE_HOME, set by + +> Sys.getenv() + +If JAVA is not correctly set, you will see an installatin error at a latter stage when loading the package `RJava`. + +Note in some case, you may need to reboot your computer to ensure that this environement variable is properly accounted for. + + 2. [Install R](https://cran.rstudio.com/): follow instruction from the installer. + + 3. **Only for windows user** [Install RTools](https://cran.r-project.org/bin/windows/Rtools/): This executable is needed to install the package from github. Follow instruction from the installer. + + 4. [Install R Studio](https://www.rstudio.com/products/rstudio/download/#download) : follow instruction from the installer + +You can now Launch __R Studio__ ### Package installation: koboloadeR from Github (up to date version): @@ -77,7 +86,7 @@ Note that the package is still in beta-version. We hope to have soon a release a * Open R studio interface and within the R console, install `devtools` package: ``` -devtools::install.packages("devtools") +install.packages("devtools") ``` * Install koboloadeR: diff --git a/buildsite.R b/buildsite.R index c5f99df..573b0b6 100644 --- a/buildsite.R +++ b/buildsite.R @@ -6,3 +6,4 @@ library("pkgdown") pkgdown::build_site() +build_vignettes() diff --git a/docs/articles/index.html b/docs/articles/index.html index c5871f7..58e1dbd 100644 --- a/docs/articles/index.html +++ b/docs/articles/index.html @@ -85,6 +85,9 @@
To be able to use koboloadeR you will need:
-Java JRE (https://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html): JAVA is required to manipulate excel files.
R (https://cran.rstudio.com/). For Windows, choose “install R for the first time”.
Only for windows user RTools (https://cran.r-project.org/bin/windows/Rtools/) - This executable is needed to install the package from github.
R Studio (https://www.rstudio.com/products/rstudio/download/#download)
If you install, the JDK
(Java Development kit), Please make sure that JAVA_HOME
is actually recorded as an Environment Variable.
If you install, the JRE
(Java Runtime Environment), Please make sure that JRE_HOME
is actually recorded as an Environment Variable.
Once in R, you may double-check that Environement variable are correctly, i.e. JAVA_HOME or JRE_HOME, set by
+++Sys.getenv()
+
If JAVA is not correctly set, you will see an installatin error at a latter stage when loading the package RJava
.
Note in some case, you may need to reboot your computer to ensure that this environement variable is properly accounted for.
+Install R: follow instruction from the installer.
Only for windows user Install RTools: This executable is needed to install the package from github. Follow instruction from the installer.
Install R Studio : follow instruction from the installer
You can now Launch R Studio
devtools
package:devtools::install.packages("devtools")
+install.packages("devtools")