19  Stacked ensembles

Stacking combines predictions from a library of base learners. Cross-validated base predictions train a meta learner; base learners are then refitted on the training data for prediction on new cases.

19.1 SuperLearner

The example combines a generalized linear model and a CART model for a numeric outcome. It requires the nnls package for the default meta learner.

dat <- set_outcome(mtcars[, c("mpg", "wt", "hp", "disp")], "mpg")
config <- setup_SuperLearner(
  base_learners = list(linear = setup_GLM(), tree = setup_CART(maxdepth = 3L)),
  inner_resampling_config = setup_KFold(n_resamples = 3L, seed = 2026L)
)
mod <- train(
  dat,
  hyperparameters = config,
  execution_config = setup_SerialExecution(seed = 2027L)
)

2026-10-08 02:53:49 Checking data is ready for training... ✔ [check_supervised]
2026-10-08 02:53:49 ▶ [train]
2026-10-08 02:53:50 Training set: 32 cases x 3 features. [summarize_supervised]
2026-10-08 02:53:50 // CPU | serial | 1 worker: algorithm 1 thread, tuning 1, outer resampling 1 [msg_resources]
2026-10-08 02:53:50 Shared memory (auto): not sharing training data (4.5 kB) -- this run transfers nothing to workers. [report_shared_memory]
2026-10-08 02:53:50 Training SuperLearner Regression... [train]
2026-10-08 02:53:50 Checking data is ready for training... ✔ [check_supervised]
2026-10-08 02:53:50 Stacking 2 library entries: linear, tree. [train_]

⠋ Cross-fitting library 1/3
⠙ Cross-fitting library 2/3 ETA 0:00
⠹ Cross-fitting library 3/3 ETA 0:00
⠸ Cross-fitting library 3/3         
                           
2026-10-08 02:53:50 ✔ Cross-fitting library 3/3 done in 0:00

<Regression>
SuperLearner (SuperLearner: cross-validated stacked ensemble)

  <Training Regression Metrics>
   MAE: 1.46
   MSE: 3.20
  RMSE: 1.79
    R²: 0.91

2026-10-08 02:53:51 Done in 1.42 seconds. [train]

get_varimp(mod)

<VariableImportance>
  weight (ensemble_weight, 2 entries): Non-negative least squares coefficient of each base learner's cross-fitted predictions in the meta learner, divided by the sum of the coefficients so that they sum to 1.
  cv_risk (cross_validated_risk, 2 entries): Mean squared error of each base learner's cross-fitted predictions against the outcome.

SuperLearner supports regression and binary classification. The default nonnegative least-squares meta learner combines the library predictions. discrete = TRUE selects a library entry. Inner cross-validation constructs the ensemble; outer resampling or a separate test sample assesses its predictive performance.

19.2 Modality stacking

setup_ModalityStacking() binds each library entry to a group of predictors. Names align the groups and their base learners.

modality_config <- setup_ModalityStacking(
  feature_groups = list(size = c("wt", "disp"), power = "hp"),
  base_learners = list(size = setup_GLM(), power = setup_CART()),
  inner_resampling_config = setup_KFold(n_resamples = 3L, seed = 2026L)
)

This configuration represents measurements from separate modalities while retaining their identity in the model specification.

19.3 Conditional SuperLearner

setup_ConditionalSuperLearner() fits an oracle that routes each case to one expert from the library. It controls the expert library, oracle learner, iteration count, and minimum region size. Its case-dependent selection has a different interpretation from a single set of global ensemble weights.

Importance measures for ensembles identify the quantity reported, such as a library weight or cross-validated risk. Inspect their metadata before comparing rankings.