A Linear Additive Tree (LINAD) fits linear models along a decision-tree path. The coefficients at a leaf are the sum of the contributions along that path. The fitted model therefore represents region-specific linear relationships.
<SupervisedReview>
Linear Additive Tree was used for regression. Hyperparameter tuning was performed using exhaustive grid search. R-squared was 0.68 on the training set and 0.29 on the test set.
100 training and 20 test cases; 3 predictors (33.3 training cases per predictor).
Performance
Training Test Training - test
MAE 0.625 1.138 -0.513
MSE 0.937 3.229 -2.292
RMSE 0.968 1.797 -0.829
R² 0.678 0.285 0.393
Baseline comparisons
MSE 3.229 vs 4.535 (training mean); loss reduction 1.305 (95% CI 0.060 to 2.550): better
MAE 1.138 vs 1.622 (training mean); loss reduction 0.484 (95% CI 0.153 to 0.815): better
R² 0.285 vs -0.004 (training mean)
Findings
* SINGLE_SPLIT (note): Performance was estimated on a single split, with 20 test cases. The test intervals reflect the number of test cases, not how much the estimate would change with a different split.
Suggestion: Use outer resampling to test on every case and to measure variation between splits; repeat it when cases are few.
* TEST_PRECISION (note): Test mean squared error is 3.229 (95% CI 0.000 to 7.192), from 20 test cases.
* BASELINE_MSE (note): Test mean squared error of 3.229 (mean reduction from the baseline's 4.535: 1.305, 95% CI 0.060 to 2.550) is better than predicting the training mean for every case.
Limitations
- Whether the model is useful depends on the question it addresses, the costs of different errors, and how its predictions will be used; performance metrics cannot establish this.
- Preprocessing, decomposition and tuning done by rtemis are fitted on the training cases of each split and only applied to its test cases. Steps taken before the data was passed to rtemis -- selecting or filtering predictors, transforming or removing cases using all the data, or choosing among models by their test performance -- can bias evaluation; this review cannot determine whether that happened.
- Performance estimates apply to cases from the population the data was drawn from; performance on a different population or setting may differ.
The example compares learning rates through internal resampling. tuner_config controls this selection procedure; see hyperparameter tuning. max_leaves bounds tree size.
17.2 Control model structure
node_model selects forward, ridge, or elastic-net node models. split_features, linear_features, and global_features specify columns that may split, enter local linear models, or have coefficients shared across leaves. Root-specific settings control the root fit. split_search selects the split-scoring procedure.
max_leaves = 1L produces a single linear model. gamma = 0 gives constant node models and a CART-equivalent tree configuration.
get_varimp(mod)
<VariableImportance>importance (standardized_coefficient, 3 entries): Absolute leaf coefficient of the predictor times its training standard deviation, averaged over the leaves of the selected tree weighted by their training cases; a factor's level columns are summed.
split_gain (split_gain, 3 entries): Reduction in training loss summed over the splits on the predictor that the selected tree reaches.
LINAD importance distinguishes linear effects from split gain. draw_linad() displays the fitted tree with node coefficient tables; get_learning_curve() and plot_learning() expose training progress.
17.3 LINAD forests
setup_LINADForest() configures a bagged ensemble of linear additive trees. Each tree selects its size using its out-of-bag cases. mtry_tree and mtry_split control feature sampling at tree and split levels. The fit reports out-of-bag predictions and metrics; se() computes infinitesimal jackknife standard errors.
