I am trying to use approximate Bayesian computation (ABC) to calibrate parameters in order to obtain a model output variable that is close to an observed value in the literature. To do this, I am using the abc package.
In a first step, I am testing the effect of the choice of tolerance rate on the estimates with ABC using the cv4abc function. However, the results show that the estimates are not accurate as they are very different from the true values. I have tested several types of ABC algorithms (rejection, loclinear, and neuralnet) and transformations (none and log). I am probably doing something wrong in my code. So any help would be greatly appreciated to understand what is causing inaccurate estimates. Or is there a more appropriate method for calibrating parameters?
Here's my data: https://www.dropbox.com/scl/fi/mzgxownug9sns8q1rlqnt/Test.csv?rlkey=8ggfob5zeu5k6zoode721e7t2&st=1nnhogcy&dl=0
The range of the observed response variable in the literature is: min = 0.5, max = 2.5, and mean = 1.5.
The simulated summary statistics in the dataset is: simulated_rmse. For each row of the dataset, I have calculated the RMSE between the simulated data (column = “simulated_response”) and the observed value in the literature (i.e., 1.5).
The simulated parameters and therefore to be estimated in the dataset are: param_1 to param_20.
The observed summary statistics is 0 (i.e. RMSE = 0)
Here is an example of a plot for a parameter using the 'rejection' method and 'none' transformation:
Here is my code:
library(abc)
data <- read.csv("C:/Users/Downloads/Test.csv")
## summary(data)
## Observed summary statistics
observed_summary_statistics <- 0
## Simulated summary statistics
simulated_summary_statistics <- data[, c("simulated_RMSE")]
## summary(simulated_summary_statistics)
## Simulated parameter values
simulated_parameters <- data[, !(colnames(data) %in% c("simulated_response", "simulated_RMSE"))]
## summary(simulated_parameters)
## Run the "cv4abc" function of the "abc" package
cv_abc_rejection <- abc::cv4abc(param = simulated_parameters, sumstat = simulated_summary_statistics, nval = 100, tols = c(0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1), method = "rejection", transf = "none")
## plot(cv_abc_rejection)
## summary(cv_abc_rejection)
cv_abc_rejection_log <- abc::cv4abc(param = simulated_parameters, sumstat = simulated_summary_statistics, nval = 100, tols = c(0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1), method = "rejection", transf = "log")
## plot(cv_abc_rejection_log)
## summary(cv_abc_rejection_log)
cv_abc_loclinear <- abc::cv4abc(param = simulated_parameters, sumstat = simulated_summary_statistics, nval = 100, tols = c(0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1), method = "loclinear", transf = "none")
## plot(cv_abc_loclinear)
## summary(cv_abc_loclinear)
cv_abc_loclinear_log <- abc::cv4abc(param = simulated_parameters, sumstat = simulated_summary_statistics, nval = 100, tols = c(0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1), method = "loclinear", transf = "log")
## plot(cv_abc_loclinear_log)
## summary(cv_abc_loclinear_log)
cv_abc_neuralnet <- abc::cv4abc(param = simulated_parameters, sumstat = simulated_summary_statistics, nval = 100, tols = c(0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1), method = "neuralnet", transf = "none")
## plot(cv_abc_neuralnet)
## summary(cv_abc_neuralnet)
cv_abc_neuralnet_log <- abc::cv4abc(param = simulated_parameters, sumstat = simulated_summary_statistics, nval = 100, tols = c(0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1), method = "neuralnet", transf = "log")
## plot(cv_abc_neuralnet_log)
## summary(cv_abc_neuralnet_log)
