bias_variance() refits a fixed learner across training resamples and compares its predictions at a fixed test set. For squared-error loss, the variation of the fitted function and its average deviation from a reference function describe variance and squared bias.
set.seed(2026L)
dat <- data.frame(x = rnorm(160))
truth <- 2 * dat[["x"]]
dat[["outcome"]] <- truth + rnorm(160)
result <- bias_variance(
dat[1:120, ],
dat_test = dat[121:160, ],
true_values = truth[121:160],
hyperparameters = setup_CART(maxdepth = 3L),
resampler_config = setup_StratBoot(n_resamples = 10L, seed = 2026L),
execution_config = setup_SerialExecution(seed = 2027L)
)result<BiasVariance>
Mean squared bias: 0.12 (0.43)
Mean variance: 0.22 (0.20)
The small number of resamples demonstrates the interface. Estimates of bias and variance themselves vary with the resampling experiment.
true_values supplies the known function at the test cases. Without it, squared deviations from observed outcomes include observation noise. For binary classification, the function decomposes squared probability error; it does not provide an additive decomposition of classification error.
