Split a dataset into training and held-out test partitions

A held-out split is a decision – which rows train, which are held out, and by what rule – and a decision the record cannot report is one nothing downstream can audit. partition() makes it a first-class, executed step, the same way ingest() makes reading a file one: the config states the rule, this function applies it and returns a manifest naming exactly what happened, including a DataFingerprint of each side.

Not a SuperConfig property. SuperConfig's dat_training_path and dat_test_path simply name whatever partition() (or any other step) produced – the same relationship they already have with ingest()'s output.

Usage

partition(dat, config, outdir = NULL, overwrite = FALSE, verbosity = 1L)

Arguments

dat: data.frame or data.table: The dataset to split.
config: PartitionConfig: The split rule.
outdir: Optional Character: Directory to write training.parquet and test.parquet to; created if it does not exist. If NULL, nothing is written and each outputs entry carries its fingerprint and row count without a path.
overwrite: Logical: If TRUE, overwrite existing output files.
verbosity: Integer: Verbosity level.

Value

Named list: the partition manifest – config, source (a DataFingerprint of dat), and outputs (a list with training and test, each holding a fingerprint and, when outdir is given, the path written to).

Author

EDG

Examples

partition(iris, setup_RandomPartition(train_p = 0.8, seed = 2026L))

#> 2026-10-20 09:07:55 
#> ▶ Partitioned 150 cases into 120 training and 30 test (random).
#>  [partition]
#> $config
#> $config$method
#> [1] "random"
#> 
#> $config$train_p
#> [1] 0.8
#> 
#> $config$seed
#> [1] 2026
#> 
#> 
#> $source
#> $source$method
#> [1] "object"
#> 
#> $source$algorithm
#> [1] "sha256"
#> 
#> $source$encoding
#> [1] "r-serialize-v3"
#> 
#> $source$hash
#> [1] "e3449b68076a4e29c103a0dae0cce63885ab920883181b2c10eddcbb1c6ffc41"
#> 
#> $source$n_rows
#> [1] 150
#> 
#> $source$n_cols
#> [1] 5
#> 
#> $source$column_names
#> [1] "Sepal.Length" "Sepal.Width"  "Petal.Length" "Petal.Width"  "Species"     
#> 
#> $source$source
#> NULL
#> 
#> $source$language
#> [1] "R"
#> 
#> $source$data_structure
#> [1] "data.frame"
#> 
#> 
#> $outputs
#> $outputs$training
#> $outputs$training$path
#> NULL
#> 
#> $outputs$training$fingerprint
#> $outputs$training$fingerprint$method
#> [1] "object"
#> 
#> $outputs$training$fingerprint$algorithm
#> [1] "sha256"
#> 
#> $outputs$training$fingerprint$encoding
#> [1] "r-serialize-v3"
#> 
#> $outputs$training$fingerprint$hash
#> [1] "a6a68f03b6f469f4e887dcc44af7bf473579a7e946e26cba3c11149aee5cf509"
#> 
#> $outputs$training$fingerprint$n_rows
#> [1] 120
#> 
#> $outputs$training$fingerprint$n_cols
#> [1] 5
#> 
#> $outputs$training$fingerprint$column_names
#> [1] "Sepal.Length" "Sepal.Width"  "Petal.Length" "Petal.Width"  "Species"     
#> 
#> $outputs$training$fingerprint$source
#> NULL
#> 
#> $outputs$training$fingerprint$language
#> [1] "R"
#> 
#> $outputs$training$fingerprint$data_structure
#> [1] "data.frame"
#> 
#> 
#> $outputs$training$n_rows
#> [1] 120
#> 
#> 
#> $outputs$test
#> $outputs$test$path
#> NULL
#> 
#> $outputs$test$fingerprint
#> $outputs$test$fingerprint$method
#> [1] "object"
#> 
#> $outputs$test$fingerprint$algorithm
#> [1] "sha256"
#> 
#> $outputs$test$fingerprint$encoding
#> [1] "r-serialize-v3"
#> 
#> $outputs$test$fingerprint$hash
#> [1] "4a07435d9edb98847fab2cafc02805046058a1e145c9ecb0289103a7ada3a3aa"
#> 
#> $outputs$test$fingerprint$n_rows
#> [1] 30
#> 
#> $outputs$test$fingerprint$n_cols
#> [1] 5
#> 
#> $outputs$test$fingerprint$column_names
#> [1] "Sepal.Length" "Sepal.Width"  "Petal.Length" "Petal.Width"  "Species"     
#> 
#> $outputs$test$fingerprint$source
#> NULL
#> 
#> $outputs$test$fingerprint$language
#> [1] "R"
#> 
#> $outputs$test$fingerprint$data_structure
#> [1] "data.frame"
#> 
#> 
#> $outputs$test$n_rows
#> [1] 30
#> 
#> 
#>