Setup hyperparameters for TabNet training.
Usage
setup_TabNet(
batch_size = 500L,
penalty = 0.001,
clip_value = NULL,
loss = "auto",
epochs = 50L,
drop_last = FALSE,
decision_width = NULL,
attention_width = NULL,
num_steps = 3L,
feature_reusage = 1.3,
mask_type = "sparsemax",
virtual_batch_size = 65536L,
valid_split = 0,
learn_rate = 0.02,
optimizer = "adam",
lr_scheduler = NULL,
lr_decay = 0.1,
step_size = 30L,
checkpoint_epochs = 10L,
cat_emb_dim = 1L,
num_independent = 2L,
num_shared = 2L,
num_independent_decoder = 1L,
num_shared_decoder = 1L,
momentum = 0.02,
pretraining_ratio = 0.5,
device = "auto",
importance_sample_size = NULL,
early_stopping_monitor = "auto",
early_stopping_tolerance = 0,
early_stopping_patience = 0L,
num_workers = 0L,
skip_importance = FALSE,
ifw = FALSE
)Arguments
- batch_size
(Tunable) Integer [1, Inf): Batch size.
- penalty
(Tunable) Numeric [0, Inf): Regularization penalty.
- clip_value
(Tunable) Optional Numeric: Clip value.
- loss
(Tunable) Character: Loss function.
- epochs
(Tunable) Integer [1, Inf): Number of epochs.
- drop_last
(Tunable) Logical: If TRUE, drop last batch.
- decision_width
(Tunable) Optional Integer [1, Inf): Decision width.
- attention_width
(Tunable) Optional Integer [1, Inf): Attention width.
- num_steps
(Tunable) Integer [1, Inf): Number of steps.
- feature_reusage
(Tunable) Numeric [0, Inf): Feature reusage.
- mask_type
(Tunable) Character {"sparsemax", "entmax"}: Mask type.
- virtual_batch_size
(Tunable) Integer [1, Inf): Virtual batch size.
- valid_split
(Tunable) Numeric [0, 1): Validation split.
- learn_rate
(Tunable) Numeric (0, Inf): Learning rate.
- optimizer
Character: Optimizer name, resolved by the tabnet backend.
- lr_scheduler
Optional Character {"step", "reduce_on_plateau"}: Learning-rate scheduler.
- lr_decay
(Tunable) Numeric [0, 1]: Learning rate decay.
- step_size
(Tunable) Integer [1, Inf): Step size.
- checkpoint_epochs
(Tunable) Integer [1, Inf): Checkpoint epochs.
- cat_emb_dim
(Tunable) Integer [1, Inf): Categorical embedding dimension.
- num_independent
(Tunable) Integer [1, Inf): Number of independent Gated Linear Units (GLU) at each step of the encoder.
(Tunable) Integer [1, Inf): Number of shared Gated Linear Units (GLU) at each step of the encoder.
- num_independent_decoder
(Tunable) Integer [1, Inf): Number of independent GLU layers for pretraining.
(Tunable) Integer [1, Inf): Number of shared GLU layers for pretraining.
- momentum
(Tunable) Numeric [0, Inf): Momentum.
- pretraining_ratio
(Tunable) Numeric [0, 1]: Pretraining ratio.
- device
Character {"auto", "cpu", "cuda"}: Compute device.
- importance_sample_size
(Tunable) Optional Integer [1, Inf): Importance sample size.
- early_stopping_monitor
(Tunable) Character {"auto", "valid_loss", "train_loss"}: Early stopping monitor.
- early_stopping_tolerance
(Tunable) Numeric [0, Inf): Minimum relative improvement to reset the patience counter.
- early_stopping_patience
(Tunable) Integer [0, Inf): Number of epochs without improving before stopping.
- num_workers
Integer [0, Inf): Number of subprocesses for data loading.
- skip_importance
Logical: If TRUE, skip importance calculation.
- ifw
(Tunable) Logical: If TRUE, use Inverse Frequency Weighting in classification.
Examples
tabnet_hyperparams <- setup_TabNet(epochs = 100L, learn_rate = 0.01)
tabnet_hyperparams
#> <TabNetHyperparameters>
#> hyperparameters:
#> batch_size: <int> 500
#> penalty: <nmr> 1e-03
#> clip_value: <NUL> NULL
#> loss: <chr> auto
#> epochs: <int> 100
#> drop_last: <lgc> FALSE
#> decision_width: <NUL> NULL
#> attention_width: <NUL> NULL
#> num_steps: <int> 3
#> feature_reusage: <nmr> 1.30
#> mask_type: <chr> sparsemax
#> virtual_batch_size: <int> 65536
#> valid_split: <nmr> 0.00
#> learn_rate: <nmr> 0.01
#> optimizer: <chr> adam
#> lr_scheduler: <NUL> NULL
#> lr_decay: <nmr> 0.10
#> step_size: <int> 30
#> checkpoint_epochs: <int> 10
#> cat_emb_dim: <int> 1
#> num_independent: <int> 2
#> num_shared: <int> 2
#> num_independent_decoder: <int> 1
#> num_shared_decoder: <int> 1
#> momentum: <nmr> 0.02
#> pretraining_ratio: <nmr> 0.50
#> device: <chr> auto
#> importance_sample_size: <NUL> NULL
#> early_stopping_monitor: <chr> auto
#> early_stopping_tolerance: <nmr> 0.00
#> early_stopping_patience: <int> 0
#> num_workers: <int> 0
#> skip_importance: <lgc> FALSE
#> ifw: <lgc> FALSE
#> tunable_hyperparameters: <chr> batch_size, penalty, clip_value, loss, epochs, drop_last, decision_width, attention_width, num_steps, feature_reusage, mask_type, virtual_batch_size, valid_split, learn_rate, lr_decay, step_size, checkpoint_epochs, cat_emb_dim, num_independent, num_shared, num_independent_decoder, num_shared_decoder, momentum, pretraining_ratio, importance_sample_size, early_stopping_monitor, early_stopping_tolerance, early_stopping_patience, ifw
#> fixed_hyperparameters: <chr> optimizer, lr_scheduler, device, num_workers, skip_importance
#> tuned: <int> -1
#> resampled: <int> 0
#> n_workers: <int> 1
#>
#> No search values defined for tunable hyperparameters.