Skip to main content

5  Read and ingest data

read() loads a dataset into R. ingest() combines reading with a typed configuration, Parquet output, and a manifest of the input and output.

5.1 Read a local file

This example creates a CSV file from a built-in dataset and reads it back.

csv_path <- tempfile(fileext = ".csv")
write.csv(iris, csv_path, row.names = FALSE)
dat <- read(csv_path)
2026-10-26 02:22:59 ▶ Reading file3dd3ea4b542.csv using data.table... [read]
2026-10-26 02:22:59 Read in 150 x 5 [read]
2026-10-26 02:22:59 Done in 3e-03 seconds. [read]
head(dat)
   Sepal_Length Sepal_Width Petal_Length Petal_Width Species
          <num>       <num>        <num>       <num>  <char>
1:          5.1         3.5          1.4         0.2  setosa
2:          4.9         3.0          1.4         0.2  setosa
3:          4.7         3.2          1.3         0.2  setosa
4:          4.6         3.1          1.5         0.2  setosa
5:          5.0         3.6          1.4         0.2  setosa
6:          5.4         3.9          1.7         0.4  setosa

Supported formats include delimited text, ARFF, Parquet, Excel, Stata, FASTA, and RDS. Format-specific backends and options are listed in ?read. Tabular supervised input requires numeric or factor columns; categorical labels can be converted explicitly to factors before training.

5.2 Read from a URL

read() and ingest() accept an http:// or https:// URL. The file is downloaded to a temporary directory, read like a local file, and deleted. The file extension in the URL path selects the reader.

dat <- read(
  "https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv"
)

The content behind a URL can change between downloads. ingest() records the URL, the time of the download, and the SHA-256 of the file in its manifest, together with a fingerprint of the data read, and writes a Parquet file that later steps read in place of the URL.

5.3 Read a Hugging Face dataset

An hf://datasets/<owner>/<name>/<file> path reads a file from a public dataset on the Hugging Face Hub, in the path form that huggingface_hub, DuckDB, and polars also read. A path that names the repository only reads its one data file.

dat <- read("hf://datasets/rtemis/tcga-pancan-hiseq")

@<revision> after the dataset name selects a branch, tag, or commit, and main applies when none is given. The revision is resolved to a commit, the file is downloaded at that commit, and its size and hash are checked against the Hub’s listing. ingest() records the path pinned to that commit, so a later run reads the same file:

ingestion <- ingest(
  "hf://datasets/rtemis/tcga-pancan-hiseq",
  "tcga-pancan-hiseq.parquet"
)
ingestion[["input_resolved"]]
#> "hf://datasets/rtemis/tcga-pancan-hiseq@44b05c18d65823f80137f8c1be7a0f7d071b5547/data/tcga-pancan-hiseq.parquet"

5.4 Record ingestion decisions

parquet_path <- tempfile(fileext = ".parquet")
ingestion <- ingest(
  csv_path,
  parquet_path,
  config = setup_DelimitedIngest(
    sep = ",",
    character2factor = TRUE,
    clean_colnames = FALSE
  )
)
2026-10-26 02:22:59 ▶ Reading file3dd3ea4b542.csv using data.table...
2026-10-26 02:22:59 Read in 150 x 5
2026-10-26 02:22:59 Converting 1 character feature to a factor... [preprocess_data]
2026-10-26 02:22:59 Preprocessing done. [preprocess_data]
2026-10-26 02:22:59 Done in 0.02 seconds.
2026-10-26 02:22:59 ▶ Ingested file3dd3ea4b542.csv to file3dd32dc89ff6.parquet [ingest]
ingestion
$input
[1] "/private/var/folders/cr/lgjpmnv97lg6qtnjt9kph7qw0000gn/T/Rtmpe3yT2Q/file3dd3ea4b542.csv"

$input_resolved
NULL

$input_sha256
[1] "d440daded18634c1da2f05e6b1a30385f2aca6cd38455b31d263e1657260112a"

$retrieved_at
NULL

$output
[1] "/var/folders/cr/lgjpmnv97lg6qtnjt9kph7qw0000gn/T//Rtmpe3yT2Q/file3dd32dc89ff6.parquet"

$format
[1] "delimited"

$config
$config$format
[1] "delimited"

$config$columns
NULL

$config$character2factor
[1] TRUE

$config$clean_colnames
[1] FALSE

$config$remove_duplicates
[1] FALSE

$config$reader
[1] "data.table"

$config$sep
[1] ","

$config$quote
[1] "\""

$config$na_strings
[1] ""


$engine
[1] "R"

$rtemis_version
[1] "1.4.3"

$data_input
$data_input$method
[1] "object"

$data_input$algorithm
[1] "sha256"

$data_input$encoding
[1] "r-serialize-v3"

$data_input$hash
[1] "68cecb3cce3aa1838f5a510a43fc0e572f257796d9682dd2bb588a5598b04b84"

$data_input$n_rows
[1] 150

$data_input$n_cols
[1] 5

$data_input$column_names
[1] "Sepal.Length" "Sepal.Width"  "Petal.Length" "Petal.Width"  "Species"     

$data_input$source
NULL

$data_input$language
[1] "R"

$data_input$data_structure
[1] "data.table"


$n_rows
[1] 150

$n_cols
[1] 5

Ingestion configurations select a reader and record column conversion, column-name cleaning, and duplicate handling. The columns argument carries explicit column type declarations. Format-specific constructors include setup_ParquetIngest(), setup_RDSIngest(), setup_XLSXIngest(), setup_DTAIngest(), and setup_ARFFIngest().

Preserve the input file, its data dictionary, and the ingestion manifest alongside the analysis. Continue with data checks.

© 2026 E.D. Gennatas