Setup hyperparameters for Highly Adaptive Lasso training.
Usage
setup_HAL(
max_degree = 2L,
smoothness_orders = 1L,
reduce_basis = NULL,
num_knots = NULL,
use_min = TRUE,
nfolds = 10L,
max_basis = 5000000L,
seed = NULL,
ifw = FALSE
)Arguments
- max_degree
(Tunable) Integer [1, Inf): Highest order of interaction among features that a basis function may represent.
- smoothness_orders
(Tunable) Integer [0, 9]: Smoothness of the basis functions: 0 fits zero-order indicators, 1 piecewise linear splines, higher values higher-order splines.
- reduce_basis
(Tunable) Optional Numeric (0, 1]: Minimum proportion of cases a basis function must be non-zero in to be kept. Applies only when
smoothness_ordersis 0; a search that also covers higher orders drops it from those grid cells.- num_knots
Optional Integer [1, Inf) vector: Number of knots per interaction degree, one value per degree, non-increasing. NULL generates them from max_degree and smoothness_orders.
- use_min
Logical: If TRUE, select
lambda.minfrom the internal cross-validation; if FALSE, the more heavily penalizedlambda.1se.- nfolds
Integer [3, Inf): Number of folds of the internal cross-validation that selects lambda.
- max_basis
Integer [1, Inf): Largest projected basis-function count that will be fit.
- seed
Optional Integer: Random seed for the internal cross-validation's fold assignment. NULL leaves it drawn from the ambient RNG.
- ifw
(Tunable) Logical: If TRUE, use Inverse Frequency Weighting in classification.
Details
Both outcome types are fit with hal9001::fit_hal, which spans the outcome
with a basis of indicator or spline terms over every interaction of the
features up to max_degree, then fits a lasso over that basis.
Regression uses the gaussian family, binary classification the binomial one;
hal9001 has no multinomial family, so multiclass classification is not
supported.
lambda is selected by cross-validation inside the fit and is not a search
dimension. seed fixes that cross-validation's fold assignment; nfolds
and use_min control it.
hal9001 takes a numeric matrix, so factors are one-hot encoded first and
the encoder is re-applied at predict time.
Scaling: the basis has one function per knot per feature subset, so its size
grows as C(n_features, max_degree) and, through the knots, with the number
of cases – making the lasso that follows quadratic in the number of cases.
Training projects the basis size up front, reports it at verbosity >= 1,
warns past a million, and aborts past max_basis. max_degree is the
strongest lever on that count, num_knots the next.
get_varimp() reports two measures, both aggregated over the basis
functions that involve each feature:
importance: the sum of the absolute values of their non-zero coefficients.max_coefficient: the largest single such absolute coefficient, separating a feature carried by one strong term from one carried by many weak ones.
plot_varimp(mod, measure = "max_coefficient") plots the second; the first
is the default. Both read coefficients on the scale of the basis functions.
At smoothness_orders = 0 the basis is made of indicators, so the
coefficients are unit-free and directly comparable across features; at
higher orders each basis function carries the units of its feature, so scale
the features first if they are not already comparable.
Examples
hal_hyperparams <- setup_HAL(max_degree = 1L)
hal_hyperparams
#> <HALHyperparameters>
#> hyperparameters:
#> max_degree: <int> 1
#> smoothness_orders: <int> 1
#> num_knots: <NUL> NULL
#> reduce_basis: <NUL> NULL
#> max_basis: <int> 5000000
#> cv_select: <lgc> TRUE
#> use_min: <lgc> TRUE
#> nfolds: <int> 10
#> seed: <NUL> NULL
#> ifw: <lgc> FALSE
#> tunable_hyperparameters: <chr> max_degree, smoothness_orders, reduce_basis, ifw
#> fixed_hyperparameters: <chr> num_knots, max_basis, use_min, nfolds, seed
#> tuned: <int> -1
#> resampled: <int> 0
#> n_workers: <int> 1
#>
#> No search values defined for tunable hyperparameters.