Classification Metrics
Usage
classification_metrics(
true_labels,
predicted_labels,
predicted_prob = NULL,
binclasspos = 2L,
calc_auc = TRUE,
calc_brier = TRUE,
auc_method = "lightAUC",
sample = NULL,
verbosity = 0L
)Arguments
- true_labels
Factor: True labels.
- predicted_labels
Factor: predicted values.
- predicted_prob
Optional Numeric vector or matrix [0, 1]: Predicted probabilities, one column per class. A single column is the positive class's probability and enables the probability-based metrics (AUC, Brier score); a multiclass matrix has no single score per case, so those are not computed.
- binclasspos
Integer: Factor level position of the positive class in binary classification.
- calc_auc
Logical: If TRUE, calculate AUC. May be slow in very large datasets.
- calc_brier
Logical: If TRUE, calculate Brier score.
- auc_method
Character: "lightAUC", "pROC", "ROCR".
- sample
Optional Character {"Training", "Validation", "Test", "Calibrated Training", "Calibrated Validation", "Calibrated Test"}: Sample the metrics describe.
- verbosity
Integer: Verbosity level.
Details
Note that auc_method = "pROC" is the only one that will output an AUC even if one or more predicted probabilities are NA.
Examples
# Assume positive class is "b"
true_labels <- factor(c("a", "a", "a", "b", "b", "b", "b", "b", "b", "b"))
predicted_labels <- factor(c("a", "b", "a", "b", "b", "a", "b", "b", "b", "a"))
predicted_prob <- c(0.3, 0.55, 0.45, 0.75, 0.57, 0.3, 0.8, 0.63, 0.62, 0.39)
classification_metrics(true_labels, predicted_labels, predicted_prob)
#> <Classification Metrics>
#> Predicted
#> Reference b a
#> b 5 2
#> a 1 2
#>
#> Overall
#> Sensitivity 0.714
#> Specificity 0.667
#> Balanced Accuracy 0.690
#> Ppv 0.833
#> Npv 0.500
#> F1 0.769
#> Accuracy 0.700
#> Auc 0.786
#> Brier Score 0.203
#>
#> Positive Class b
classification_metrics(true_labels, predicted_labels, 1 - predicted_prob, binclasspos = 1L)
#> <Classification Metrics>
#> Predicted
#> Reference a b
#> a 2 1
#> b 2 5
#>
#> Overall
#> Sensitivity 0.667
#> Specificity 0.714
#> Balanced Accuracy 0.690
#> Ppv 0.500
#> Npv 0.833
#> F1 0.571
#> Accuracy 0.700
#> Auc 0.786
#> Brier Score 0.203
#>
#> Positive Class a