Bagging fits models to resampled training data and aggregates their predictions. Random forests also sample predictors during tree construction. The resulting behavior depends on the base learner, resampling design, and aggregation rule.
2026-10-08 02:53:35 Checking data is ready for training... ✔ [check_supervised]
2026-10-08 02:53:35 ▶ [train]2026-10-08 02:53:35 Training set: 150 cases x 4 features. [summarize_supervised]2026-10-08 02:53:35 //CPU | serial | 1 worker: algorithm 1 thread, tuning 1, outer resampling 1 (as set) [msg_resources]2026-10-08 02:53:35 Shared memory (auto): not sharing training data (7.3 kB) -- this run transfers nothing to workers. [report_shared_memory]2026-10-08 02:53:35 <> Training Ranger Classification using 3 independent folds... [train]⠋ Outer resamples 1/3⠙ Outer resamples 2/3ETA 0:02⠹ Outer resamples 3/3ETA 0:01⠸ Outer resamples 3/32026-10-08 02:53:37 ✔ Outer resamples 3/3 done in 0:01
2026-10-08 02:53:37 </> Outer resampling done. [train]
<Resampled Classification Model>Ranger (Ranger random forest)
⟳ Tested using 3 independent folds.
<Resampled Classification Training Metrics>Aggregate Confusion Matrix across resamples.
Predicted Referencesetosa versicolor virginica setosa 100 0 0
versicolor 0 95 5
virginica 0 2 98
Showing mean (sd) across resamples.
Balanced Accuracy: 0.977 (0.015) F1: 0.977 (0.015) Accuracy: 0.977 (0.015)<Resampled Classification Test Metrics>Aggregate Confusion Matrix across resamples.
Predicted Referencesetosa versicolor virginica setosa 50 0 0
versicolor 0 47 3
virginica 0 3 47
Showing mean (sd) across resamples.
Balanced Accuracy: 0.960 (0.034) F1: 0.960 (0.034) Accuracy: 0.960 (0.034)
2026-10-08 02:53:37 Done in 1.85 seconds. [train]
review(mod)
<SupervisedReview>
Ranger random forest was used for classification. Mean balanced accuracy was 0.98 in the training set and 0.96 in the test set across 3 independent folds.
3 resamples: 150 out-of-sample predictions of 150 cases, at least 99 training cases per resample; 4 predictors (8.2 minority-class training cases per predictor).
Performance, mean (SD) over 3 resamples
Training Test Training - test
Balanced Accuracy 0.977 (0.015) 0.960 (0.034) 0.016
F1 0.977 (0.015) 0.960 (0.034) 0.017
Accuracy 0.977 (0.015) 0.960 (0.034) 0.016
Baseline comparisons, mean over resamples
Accuracy 0.960 vs 0.333 (most common training class); better in 3 of 3 resamples compared
Balanced Accuracy 0.960 vs 0.333 (chance level); better in 3 of 3 resamples compared
Findings
* FOLD_VARIATION (note): Test balanced accuracy ranged from 0.922 to 0.980 across 3 resamples (mean 0.960, SD 0.034); 3 of 3 resamples in which the metric is defined outperformed their baseline. Resamples share training cases, so their results are not independent, and no confidence interval or test is computed from them.
* FEW_CASES_PER_PREDICTOR (note): There are 8.250 minority-class training cases per learner column (4 columns), fewer than 10. This is a rule of thumb from logistic regression (events per variable), not a validated sample-size requirement for every algorithm.
Suggestion: Judge the model by held-out performance. Consider a regularized or sparse algorithm, or a decomposition step in train(), which is fitted on training cases only.
* PRESELECTION_RISK (note): With few cases per predictor, predictors are often selected before modeling. Selection or filtering of predictors done on all the data before it was passed to train() biases evaluation; this review cannot determine whether that happened. Preprocessing, decomposition and tuning within train() use training cases only.
Suggestion: Avoid selecting predictors on all the data before train(). To reduce the predictors, use a sparse algorithm or a decomposition step in train(); any selection done outside rtemis must use only the training cases of each split.
Limitations
- Whether the model is useful depends on the question it addresses, the costs of different errors, and how its predictions will be used; performance metrics cannot establish this.
- Preprocessing, decomposition and tuning done by rtemis are fitted on the training cases of each split and only applied to its test cases. Steps taken before the data was passed to rtemis -- selecting or filtering predictors, transforming or removing cases using all the data, or choosing among models by their test performance -- can bias evaluation; this review cannot determine whether that happened.
- Performance estimates apply to cases from the population the data was drawn from; performance on a different population or setting may differ.
get_varimp(mod)
$Fold_1
<VariableImportance>importance (permutation, 4 entries): Increase in prediction error on each tree's out-of-bag cases when the predictor's values are permuted, averaged over trees (ranger).
$Fold_2
<VariableImportance>importance (permutation, 4 entries): Increase in prediction error on each tree's out-of-bag cases when the predictor's values are permuted, averaged over trees (ranger).
$Fold_3
<VariableImportance>importance (permutation, 4 entries): Increase in prediction error on each tree's out-of-bag cases when the predictor's values are permuted, averaged over trees (ranger).
Ranger controls tree count, predictor sampling, node size, and the importance measure. importance = "none" produces a fit without variable importance. The appropriate splitting rule and task support are documented in ?setup_Ranger.
15.2 Other forest configurations
setup_LightRF() fits the random-forest mode of LightGBM. setup_LINADForest() uses linear additive trees, with tree- and split-level feature sampling. These forests share the train() interface while retaining method-specific settings and importance measures.
Resampled model predictions can be aggregated with predict(type = "avg"). Outer resampling primarily supplies assessment of the fitting procedure; the interpretation of its aggregate is determined by that resampling design.
