Decomposition Metrics

Quantify a fitted decomposition: how much of the data its components reconstruct, how redundant they are, and – given data the fit never saw – whether it generalizes.

Usage

decomp_metrics(
  decom,
  x,
  new_data = NULL,
  execution_config = NULL,
  verbosity = 1L
)

Arguments

decom

Decomposition object.

x

Tabular data (data.frame, data.table, or tibble): The data decom was fitted on.

new_data

Optional tabular data: Data the fit never saw. Its columns must match those decom was fitted on.

execution_config

Optional ExecutionConfig object: Threads and device for encoding and reconstructing, as in apply_decomp. NULL uses the host's default worker count.

verbosity

Integer: Verbosity level.

Value

DecompositionMetrics object.

Details

Which metrics are computed follows from the algorithm's traits; see applicable_metrics. A metric its algorithm cannot support is NA.

The unprefixed metrics describe the fitted data, so x must be that data: they are computed against the components decom already holds. A different number of cases is an error, and x failing to match decom@data_fingerprint is reported without raising, since scoring a fit against an altered copy of its training data is a legitimate thing to do.

decomp() already stores this on its result for the data it was fitted on, so calling this without new_data recomputes what decom@metrics holds. The reason to call it is new_data: reconstruction_gap is the difference between out-of-sample and in-sample relative reconstruction error, and it is the signal that a decomposition has fitted its training cases rather than their structure.

Reconstruction is measured in the units of the data as it was passed to decomp(), so the numbers are comparable across algorithms and across configurations of one algorithm. One caveat: ICA with row_norm = TRUE standardizes each case across features, and inverting that needs each case's own mean and standard deviation, taken from the data being reconstructed. Its reconstruction therefore draws on information the components do not carry, and its error is not comparable with an algorithm that reconstructs from components alone.

Author

EDG

Examples

x <- exc(iris, "Species")
iris_pca <- decomp(x, algorithm = "PCA", verbosity = 0L)
decomp_metrics(iris_pca, x, verbosity = 0L)

#> <Training Decomposition Metrics>
#>      Explained Variance Ratio: 0.99
#> Relative Reconstruction Error: 0.03
#>           Reconstruction Rmse: 0.11
#> Max Abs Component Correlation: 5.8e-16
#>      Effective Dimensionality: 1.69