Setup LightGBM Hyperparameters

Setup hyperparameters for LightGBM training.

Usage

setup_LightGBM(
  max_nrounds = 1000L,
  force_nrounds = NULL,
  early_stopping_rounds = 10L,
  num_leaves = 8L,
  max_depth = -1L,
  learning_rate = 0.01,
  feature_fraction = 1,
  bagging_fraction = 1,
  bagging_freq = 1L,
  lambda_l1 = 0,
  lambda_l2 = 0,
  max_cat_threshold = 32L,
  min_data_per_group = 32L,
  linear_tree = FALSE,
  ifw = FALSE,
  objective = NULL,
  early_stopping_min_delta = 0,
  deterministic = FALSE,
  objective_seed = 5L,
  force_row_wise = FALSE,
  histogram_pool_size = -1,
  top_k = NULL,
  pos_bagging_fraction = 1,
  neg_bagging_fraction = 1,
  bagging_seed = 3L,
  feature_fraction_bynode = 1,
  feature_fraction_seed = 2L,
  cegb_tradeoff = 1,
  cegb_penalty_split = 0,
  cegb_penalty_feature_lazy = NULL,
  cegb_penalty_feature_coupled = NULL,
  use_quantized_grad = FALSE,
  num_grad_quant_bins = NULL,
  quant_train_renew_leaf = NULL,
  stochastic_rounding = NULL,
  monotone_constraints = NULL,
  monotone_constraints_method = NULL,
  monotone_penalty = NULL,
  interaction_constraints = NULL,
  feature_contri = NULL,
  min_data_in_leaf = 20L,
  min_sum_hessian_in_leaf = 0.001,
  min_gain_to_split = 0,
  max_delta_step = 0,
  path_smooth = 0,
  extra_trees = FALSE,
  extra_seed = 6L,
  linear_lambda = NULL,
  max_bin = 255L,
  min_data_in_bin = 3L,
  use_missing = TRUE,
  zero_as_missing = FALSE,
  cat_l2 = 10,
  cat_smooth = 10,
  max_cat_to_onehot = 4L,
  boosting = "gbdt",
  data_sample_strategy = "bagging",
  drop_rate = NULL,
  max_drop = NULL,
  skip_drop = NULL,
  uniform_drop = NULL,
  xgboost_dart_mode = NULL,
  drop_seed = NULL,
  top_rate = NULL,
  other_rate = NULL,
  alpha = NULL,
  tweedie_variance_power = NULL,
  fair_c = NULL,
  poisson_max_delta_step = NULL,
  sigmoid = NULL,
  boost_from_average = NULL,
  reg_sqrt = NULL,
  device_type = NULL,
  tree_learner = "serial",
  force_col_wise = TRUE
)

Arguments

max_nrounds

Integer [1, Inf): Maximum number of boosting rounds.

force_nrounds

Optional Integer [1, Inf): Use this many boosting rounds. Disables search for nrounds. NULL selects nrounds during tuning from the best iteration each tuning fit reports, up to max_nrounds: the fits stop early on their validation cases except under "dart" boosting, and a fit that reports no best iteration counts as 100 rounds.

early_stopping_rounds

Integer [1, Inf): Number of rounds without improvement to stop training.

num_leaves

(Tunable) Integer [1, Inf): Maximum number of leaves in one tree.

max_depth

(Tunable) Integer: Maximum depth of trees. -1 = no limit.

learning_rate

(Tunable) Numeric (0, 1]: Learning rate.

feature_fraction

(Tunable) Numeric (0, 1]: Fraction of features to use.

bagging_fraction

(Tunable) Numeric (0, 1]: Fraction of cases sampled per tree.

bagging_freq

(Tunable) Integer [1, Inf): Resample every this many iterations.

lambda_l1

(Tunable) Numeric [0, Inf): L1 regularization.

lambda_l2

(Tunable) Numeric [0, Inf): L2 regularization.

max_cat_threshold

(Tunable) Integer [1, Inf): Maximum number of categories for categorical features.

min_data_per_group

(Tunable) Integer [1, Inf): Minimum number of observations per categorical group.

linear_tree

(Tunable) Logical: If TRUE, use linear trees.

ifw

(Tunable) Logical: If TRUE, use Inverse Frequency Weighting in classification.

objective

Optional Character: Objective function. NULL = set from outcome type.

early_stopping_min_delta

(Tunable) Numeric [0, Inf): Smallest improvement that resets the early-stopping counter.

deterministic

Logical: Force reproducible results at some cost in speed.

objective_seed

Integer: Random seed for the objective, where it draws.

force_row_wise

Logical: Force row-wise histogram building. Cheaper in memory than column-wise, slower per iteration.

histogram_pool_size

Numeric: Memory cap in MB for the histogram cache; -1 leaves it uncapped.

top_k

(Tunable) Optional Integer [1, Inf): Features each machine votes for in the voting tree learner. Requires tree_learner = "voting". NULL uses LightGBM's default of 20.

pos_bagging_fraction

(Tunable) Numeric (0, 1]: Fraction of positive cases sampled per tree, for a binary outcome.

neg_bagging_fraction

(Tunable) Numeric (0, 1]: Fraction of negative cases sampled per tree, for a binary outcome.

bagging_seed

Integer: Random seed for bagging.

feature_fraction_bynode

(Tunable) Numeric (0, 1]: Fraction of features sampled at each node, on top of the per-tree fraction.

feature_fraction_seed

Integer: Random seed for feature sampling.

cegb_tradeoff

(Tunable) Numeric [0, Inf): Overall multiplier on every cost-efficiency penalty. Its scale is not comparable between boosting modes.

cegb_penalty_split

(Tunable) Numeric [0, Inf): Cost charged per split, in units of loss reduction.

cegb_penalty_feature_lazy

Optional List: Per-feature cost charged the first time a case uses it, in [0, Inf). One vector applied to every tree, or one vector per tree. NULL charges no cost.

cegb_penalty_feature_coupled

Optional List: Per-feature cost charged the first time any case uses it, in [0, Inf). One vector applied to every tree, or one vector per tree. NULL charges no cost.

use_quantized_grad

Logical: Discretize gradients before finding splits. Faster, and changes the fit.

num_grad_quant_bins

(Tunable) Optional Integer [2, Inf): Bins the gradients are discretized into. Requires use_quantized_grad = TRUE. NULL uses LightGBM's default of 4.

quant_train_renew_leaf

Optional Logical: Recompute leaf values from the exact gradients after each iteration. Requires use_quantized_grad = TRUE. NULL uses LightGBM's default of FALSE.

stochastic_rounding

Optional Logical: Round gradients stochastically rather than to nearest. Requires use_quantized_grad = TRUE. NULL uses LightGBM's default of TRUE.

monotone_constraints

Optional Integer [-1, 1] vector: Per-feature monotonicity: 1 increasing, -1 decreasing, 0 unconstrained. One value per feature. NULL constrains no feature.

monotone_constraints_method

Optional Character {"basic", "intermediate", "advanced"}: How monotonicity is enforced. Requires monotone_constraints. NULL uses LightGBM's default, "basic".

monotone_penalty

(Tunable) Optional Numeric [0, Inf): Depth penalty on splits that would break monotonicity. Requires monotone_constraints. NULL uses LightGBM's default of 0.

interaction_constraints

Optional Character: Feature groups permitted to interact, in LightGBM's own bracketed form, e.g. "[0,1],[2,3]". NULL permits every interaction.

feature_contri

Optional Numeric vector: Per-feature multiplier on split gain, one value per feature. NULL uses 1 for every feature.

min_data_in_leaf

(Tunable) Integer [0, Inf): Minimum number of cases in a leaf.

min_sum_hessian_in_leaf

(Tunable) Numeric [0, Inf): Minimum sum of hessians in a leaf: the smooth counterpart of a case count.

min_gain_to_split

(Tunable) Numeric [0, Inf): Minimum loss reduction a split must buy to be made.

max_delta_step

(Tunable) Numeric: Cap on each leaf's output; 0 or less leaves it uncapped.

path_smooth

(Tunable) Numeric [0, Inf): Smooth each leaf towards its parent, more strongly the fewer cases it holds.

extra_trees

(Tunable) Logical: Choose one split threshold per feature at random rather than the best.

extra_seed

Integer: Random seed for the thresholds extra_trees draws.

linear_lambda

(Tunable) Optional Numeric [0, Inf): L2 penalty on the linear models fitted at the leaves. Requires linear_tree = TRUE. NULL uses LightGBM's default of 0.

max_bin

(Tunable) Integer [2, Inf): Most bins a feature is discretized into. Fewer is faster and more regularized.

min_data_in_bin

(Tunable) Integer [1, Inf): Minimum number of cases per bin.

use_missing

Logical: Handle missing values. FALSE ignores them.

zero_as_missing

Logical: Treat zeros as missing, rather than only NA.

cat_l2

(Tunable) Numeric [0, Inf): L2 regularization on a categorical split.

cat_smooth

(Tunable) Numeric [0, Inf): Smoothing over categorical levels, which reduces the effect of rare ones.

max_cat_to_onehot

(Tunable) Integer [1, Inf): Below this many levels, split one-versus-rest rather than by partition.

boosting

Character {"gbdt", "rf", "dart"}: Boosting algorithm: gradient boosting, random forest, or DART (dropout).

data_sample_strategy

Character {"bagging", "goss"}: How cases are subsampled: at random, or by gradient magnitude (GOSS). GOSS cannot be combined with bagging, so it requires bagging_fraction = 1.

drop_rate

(Tunable) Optional Numeric [0, 1]: Fraction of existing trees to drop per iteration. Requires boosting = "dart". NULL uses LightGBM's default of 0.1.

max_drop

(Tunable) Optional Integer: Most trees dropped in one iteration; 0 or less removes the cap. Requires boosting = "dart". NULL uses LightGBM's default of 50.

skip_drop

(Tunable) Optional Numeric [0, 1]: Probability of skipping dropout for an iteration. Requires boosting = "dart". NULL uses LightGBM's default of 0.5.

uniform_drop

(Tunable) Optional Logical: Drop trees uniformly rather than weighted. Requires boosting = "dart". NULL uses LightGBM's default of FALSE.

xgboost_dart_mode

(Tunable) Optional Logical: Use the XGBoost DART normalization. Requires boosting = "dart". NULL uses LightGBM's default of FALSE.

drop_seed

Optional Integer: Random seed for choosing which trees to drop. Requires boosting = "dart". NULL uses LightGBM's default of 4.

top_rate

(Tunable) Optional Numeric [0, 1]: Share of large-gradient cases GOSS retains. Requires data_sample_strategy = "goss". NULL uses LightGBM's default of 0.2.

other_rate

(Tunable) Optional Numeric [0, 1]: Share of the remaining cases GOSS retains; with top_rate it must not exceed 1. Requires data_sample_strategy = "goss". NULL uses LightGBM's default of 0.1.

alpha

(Tunable) Optional Numeric (0, Inf): Huber delta or quantile level. Requires objective to be "huber" or "quantile". NULL uses LightGBM's default of 0.9.

tweedie_variance_power

(Tunable) Optional Numeric [1, 2): Tweedie power: 1 is Poisson-like, near 2 is gamma-like. Requires objective = "tweedie". NULL uses LightGBM's default of 1.5.

fair_c

(Tunable) Optional Numeric (0, Inf): Fair loss scale. Requires objective = "fair". NULL uses LightGBM's default of 1.

poisson_max_delta_step

(Tunable) Optional Numeric (0, Inf): Step cap safeguarding Poisson optimization. Requires objective = "poisson". NULL uses LightGBM's default of 0.7.

sigmoid

(Tunable) Optional Numeric (0, Inf): Sigmoid slope, for a binary or one-vs-all objective. NULL uses LightGBM's default of 1.

boost_from_average

Optional Logical: Start boosting from the constant score that best fits the outcome under the objective, such as its mean for regression. NULL uses LightGBM's default of TRUE.

reg_sqrt

Optional Logical: Fit the square root of the outcome and square the prediction back, for the "regression", "regression_l1", "fair", "quantile" or "mape" objective. NULL uses LightGBM's default of FALSE.

device_type

Optional Character: Deprecated and ignored. Set the compute device in the execution_config passed to train, e.g. setup_FutureExecution(device = "cuda"); "gpu" is device = "opencl" there.

tree_learner

Character {"serial", "feature", "data", "voting"}: Tree learner type.

force_col_wise

Logical: Use only with CPU - If TRUE, force col-wise histogram building.

Value

LightGBMHyperparameters object.

Details

Get more information from lightgbm::lgb.train. nrounds is auto-tuned using early stopping (up to max_nrounds) unless force_nrounds is set.

Author

EDG

Examples

lightgbm_hyperparams <- setup_LightGBM(
  max_nrounds = 500L,
  learning_rate = tune_over(0.001, 0.01, 0.05), ifw = TRUE
)
lightgbm_hyperparams

#> <LightGBMHyperparameters>
#>         hyperparameters: 
#>                                       max_nrounds: <int> 500
#>                             early_stopping_rounds: <int> 10
#>                                        num_leaves: <int> 8
#>                                         max_depth: <int> -1
#>                                     learning_rate: <tune> 0.001, 0.01, 0.05
#>                                  feature_fraction: <nmr> 1.00
#>                                  bagging_fraction: <nmr> 1.00
#>                                      bagging_freq: <int> 1
#>                                         lambda_l1: <nmr> 0.00
#>                                         lambda_l2: <nmr> 0.00
#>                                 max_cat_threshold: <int> 32
#>                                min_data_per_group: <int> 32
#>                                       linear_tree: <lgc> FALSE
#>                                               ifw: <lgc> TRUE
#>                                      tree_learner: <chr> serial
#>                                    force_col_wise: <lgc> TRUE
#>                                          boosting: <chr> gbdt
#>                              data_sample_strategy: <chr> bagging
#>                                  min_data_in_leaf: <int> 20
#>                           min_sum_hessian_in_leaf: <nmr> 1e-03
#>                                 min_gain_to_split: <nmr> 0.00
#>                                    max_delta_step: <nmr> 0.00
#>                                       path_smooth: <nmr> 0.00
#>                                       extra_trees: <lgc> FALSE
#>                                        extra_seed: <int> 6
#>                                           max_bin: <int> 255
#>                                   min_data_in_bin: <int> 3
#>                                       use_missing: <lgc> TRUE
#>                                   zero_as_missing: <lgc> FALSE
#>                                            cat_l2: <nmr> 10.00
#>                                        cat_smooth: <nmr> 10.00
#>                                 max_cat_to_onehot: <int> 4
#>                                     deterministic: <lgc> FALSE
#>                                    objective_seed: <int> 5
#>                                    force_row_wise: <lgc> FALSE
#>                               histogram_pool_size: <nmr> -1.00
#>                              pos_bagging_fraction: <nmr> 1.00
#>                              neg_bagging_fraction: <nmr> 1.00
#>                                      bagging_seed: <int> 3
#>                           feature_fraction_bynode: <nmr> 1.00
#>                             feature_fraction_seed: <int> 2
#>                                     cegb_tradeoff: <nmr> 1.00
#>                                cegb_penalty_split: <nmr> 0.00
#>                                use_quantized_grad: <lgc> FALSE
#>                          early_stopping_min_delta: <nmr> 0.00
#>                   unset: <chr> force_nrounds, objective, nrounds, best_iter, alpha, tweedie_variance_power, (+25 more)
#> tunable_hyperparameters: <int> 45
#>   fixed_hyperparameters: <int> 31
#>                   tuned: <int> 0
#>               resampled: <int> 0
#>               n_workers: <int> 1
#> 
#>   Hyperparameters learning_rate and nrounds need tuning.