Setup hyperparameters for TabNet training.
Usage
setup_TabNet(
batch_size = 500L,
penalty = 0.001,
clip_value = NULL,
loss = "auto",
epochs = 50L,
drop_last = FALSE,
decision_width = NULL,
attention_width = NULL,
num_steps = 3L,
feature_reusage = 1.3,
mask_type = "sparsemax",
virtual_batch_size = 65536L,
valid_split = 0,
learn_rate = 0.02,
optimizer = "adam",
lr_scheduler = NULL,
lr_decay = 0.1,
step_size = 30L,
checkpoint_epochs = 10L,
cat_emb_dim = 1L,
num_independent = 2L,
num_shared = 2L,
num_independent_decoder = 1L,
num_shared_decoder = 1L,
momentum = 0.02,
pretraining_ratio = 0.5,
device = NULL,
importance_sample_size = NULL,
early_stopping_monitor = "auto",
early_stopping_tolerance = 0,
early_stopping_patience = 0L,
num_workers = 0L,
skip_importance = FALSE,
ifw = FALSE
)Arguments
- batch_size
(Tunable) Integer [1, Inf): Batch size.
- penalty
(Tunable) Numeric [0, Inf): Regularization penalty.
- clip_value
(Tunable) Optional Numeric: Gradient clip value. NULL clips no gradients.
- loss
(Tunable) Character: Loss function.
- epochs
(Tunable) Integer [1, Inf): Number of epochs.
- drop_last
(Tunable) Logical: If TRUE, drop last batch.
- decision_width
(Tunable) Optional Integer [1, Inf): Decision prediction layer width. NULL uses
attention_width, or 8 where both are NULL.- attention_width
(Tunable) Optional Integer [1, Inf): Attention embedding width. NULL uses
decision_width, or 8 where both are NULL.- num_steps
(Tunable) Integer [1, Inf): Number of steps.
- feature_reusage
(Tunable) Numeric [0, Inf): Feature reusage.
- mask_type
(Tunable) Character {"sparsemax", "entmax"}: Mask type.
- virtual_batch_size
(Tunable) Integer [1, Inf): Virtual batch size.
- valid_split
(Tunable) Numeric [0, 1): Validation split.
- learn_rate
(Tunable) Numeric (0, Inf): Learning rate.
- optimizer
Character: Optimizer name, resolved by the tabnet backend.
- lr_scheduler
Optional Character {"step", "reduce_on_plateau"}: Learning-rate scheduler.
- lr_decay
(Tunable) Numeric [0, 1]: Learning rate decay.
- step_size
(Tunable) Integer [1, Inf): Step size.
- checkpoint_epochs
(Tunable) Integer [1, Inf): Checkpoint epochs.
- cat_emb_dim
(Tunable) Integer [1, Inf): Categorical embedding dimension.
- num_independent
(Tunable) Integer [1, Inf): Number of independent Gated Linear Units (GLU) at each step of the encoder.
(Tunable) Integer [1, Inf): Number of shared Gated Linear Units (GLU) at each step of the encoder.
- num_independent_decoder
(Tunable) Integer [1, Inf): Number of independent GLU layers for pretraining.
(Tunable) Integer [1, Inf): Number of shared GLU layers for pretraining.
- momentum
(Tunable) Numeric [0, Inf): Momentum.
- pretraining_ratio
(Tunable) Numeric [0, 1]: Pretraining ratio.
- device
Optional Character: Deprecated and ignored. Set the compute device in the
execution_configpassed to train, e.g.setup_FutureExecution(device = "mps"). Unset there, TabNet uses"cuda"where available and"cpu"otherwise; the Apple silicon GPU, slower than the CPU for TabNet at every size rtemis benchmarked, runs only when named.- importance_sample_size
(Tunable) Optional Integer [1, Inf): Cases drawn, with replacement, from the training cases left after the validation split to compute feature importance. Unset draws as many as there are such cases, at most 100,000. Applies only when
skip_importanceis FALSE.- early_stopping_monitor
(Tunable) Character {"auto", "valid_loss", "train_loss"}: Early stopping monitor.
- early_stopping_tolerance
(Tunable) Numeric [0, Inf): Minimum relative improvement to reset the patience counter.
- early_stopping_patience
(Tunable) Integer [0, Inf): Number of epochs without improving before stopping.
- num_workers
Integer [0, Inf): Number of subprocesses for data loading.
- skip_importance
Logical: If TRUE, skip importance calculation.
- ifw
(Tunable) Logical: If TRUE, use Inverse Frequency Weighting in classification.
Examples
tabnet_hyperparams <- setup_TabNet(epochs = 100L, learn_rate = 0.01)
tabnet_hyperparams#> <TabNetHyperparameters>
#> hyperparameters:
#> batch_size: <int> 500
#> penalty: <nmr> 1e-03
#> loss: <chr> auto
#> epochs: <int> 100
#> drop_last: <lgc> FALSE
#> num_steps: <int> 3
#> feature_reusage: <nmr> 1.30
#> mask_type: <chr> sparsemax
#> virtual_batch_size: <int> 65536
#> valid_split: <nmr> 0.00
#> learn_rate: <nmr> 0.01
#> optimizer: <chr> adam
#> lr_decay: <nmr> 0.10
#> step_size: <int> 30
#> checkpoint_epochs: <int> 10
#> cat_emb_dim: <int> 1
#> num_independent: <int> 2
#> num_shared: <int> 2
#> num_independent_decoder: <int> 1
#> num_shared_decoder: <int> 1
#> momentum: <nmr> 0.02
#> pretraining_ratio: <nmr> 0.50
#> early_stopping_monitor: <chr> auto
#> early_stopping_tolerance: <nmr> 0.00
#> early_stopping_patience: <int> 0
#> num_workers: <int> 0
#> skip_importance: <lgc> FALSE
#> ifw: <lgc> FALSE
#> unset: <chr> clip_value, decision_width, attention_width, lr_scheduler, importance_sample_size
#> tunable_hyperparameters: <int> 29
#> fixed_hyperparameters: <int> 4
#> tuned: <int> -1
#> resampled: <int> 0
#> n_workers: <int> 1
#>
#> No search values defined for tunable hyperparameters.