Get metrics

The performance metrics stored on a fitted model, clustering, or decomposition, for one sample.

Usage

get_metrics(x, sample = NULL, calibrated = FALSE)

Arguments

x

Supervised, SupervisedRes, Clustering, or Decomposition object.

sample

Optional Character {"training", "validation", "test"}: The sample whose metrics to return. Required for Supervised and SupervisedRes objects.

calibrated

Logical: If TRUE, return the metrics of the calibrated predictions of a calibrated classifier.

Value

Metrics object for a Supervised model; MetricsRes object for a SupervisedRes model; ClusteringMetrics or DecompositionMetrics object, or NULL when the object holds no metrics, for a Clustering or Decomposition.

Details

A Supervised model stores metrics for the training sample, and for the validation and test samples when train received dat_validation and dat_test. A SupervisedRes model stores per-resample metrics for the training and test samples of each outer resample, with their mean and standard deviation. A calibrated classifier also stores the metrics of its calibrated predictions, selected with calibrated = TRUE.

A Clustering or Decomposition stores metrics for the data it was fitted on, which sample = NULL and sample = "training" both select. For a decomposition, decomp_metrics computes the metrics of new data.

Requesting a sample the object does not store raises an error that names the samples it does store.

The returned object supports $ and [[: for example, get_metrics(mod, "test")$rsq.

Author

EDG

Examples

x <- data.frame(age = rnorm(100), bmi = rnorm(100))
x[["y"]] <- x[["age"]] * 2 + rnorm(100, sd = 0.3)
mod <- train(
  x[1:80, ],
  dat_test = x[81:100, ],
  hyperparameters = setup_GLM(),
  verbosity = 0L
)
get_metrics(mod, "training")

#> <Training Regression Metrics>
#>  MAE: 0.25
#>  MSE: 0.09
#> RMSE: 0.30
#>   R²: 0.98

get_metrics(mod, "test")$rsq

#> [1] 0.9868456