library(croissant)3 Reading and writing
3.1 Reading
read_croissant(x) reads one Croissant document into a Croissant object. x is one of:
- a path to a JSON-LD file;
- an
http(s)URL; - a JSON string;
- an
hf://datasets/path, described in Hugging Face datasets.
json <- '{
"@context": {
"@vocab": "https://schema.org/",
"sc": "https://schema.org/",
"cr": "http://mlcommons.org/croissant/",
"dct": "http://purl.org/dc/terms/",
"conformsTo": "dct:conformsTo",
"recordSet": "cr:recordSet",
"field": "cr:field",
"dataType": {"@id": "cr:dataType", "@type": "@vocab"}
},
"@type": "sc:Dataset",
"name": "minimal",
"conformsTo": "http://mlcommons.org/croissant/1.1",
"recordSet": [{
"@type": "cr:RecordSet",
"@id": "rs",
"field": [{"@type": "cr:Field", "@id": "rs/x", "dataType": "sc:Integer"}]
}]
}'
x <- read_croissant(json)
x<Croissant>
name: minimal
version: Croissant 1.1
distribution: none
record sets: 1
rs: 1 field
3.1.1 Versions
croissant reads Croissant 1.0 and 1.1. The version is the one named by the first conformsTo value of the form http(s)://mlcommons.org/croissant/<version>, and is stored in x@version.
| Document | Result |
|---|---|
conformsTo names 1.0 or 1.1 |
Read as that version |
No Croissant conformsTo value |
Read as 1.0, with a warning |
conformsTo names another version |
Error of class croissant_version_error |
Croissant 0.8 (no conformsTo, ml bound to http://mlcommons.org/schema/) |
Error of class croissant_version_error |
3.1.2 JSON-LD processing
The JSON-LD @context of a document maps its keys to IRIs. A document is read directly with jsonlite when its @context is a JSON object whose every entry equals the entry of the same name in the standard Croissant 1.1 context, and no nested node has its own @context. The order of entries and omitted entries do not matter.
Any other document is first compacted against the standard context with jsonld, which must then be installed; without it, read_croissant() raises an error of class rtemis_dependency_error. The following document uses a schema: prefix and full IRIs in place of the standard terms:
aliased <- '{
"@context": {
"schema": "https://schema.org/",
"dct": "http://purl.org/dc/terms/"
},
"@type": "schema:Dataset",
"schema:name": "aliased",
"dct:conformsTo": "http://mlcommons.org/croissant/1.1",
"schema:license": "CC0-1.0"
}'
y <- read_croissant(aliased)
y@name[1] "aliased"
y@license[1] "CC0-1.0"
On both paths, each key is rewritten to its canonical form: the term that the standard context defines for the key’s IRI, the local name of a schema.org IRI, or a compact IRI such as rai:dataCollection or prov:wasDerivedFrom. Values of @type and dataType become compact IRIs such as sc:Text.
Keys expand against the document’s own context, as in any JSON-LD processor. A Croissant term that the context does not define expands through @vocab to schema.org and is not read as the Croissant property. For example, dataType without a definition is https://schema.org/dataType. Each such key is reported as a warning (check undefined_term).
3.1.3 Problems found while reading
read_croissant() records every problem in a document before it returns. Three kinds of problem prevent reading:
- a value of the wrong JSON type for a typed property, such as a number for
name; - a node whose
@typedoes not include the type of its class; - a node that is not a JSON object.
When any of these occur, read_croissant() raises one error of class croissant_read_error that lists all of them. Its issues element holds the problems as a CroissantIssues object.
wrong <- sub('"name": "minimal"', '"name": 42', json)
read_croissant(wrong)Error in `read_croissant()`:
! The document has 1 problem that prevents reading it:
- [Croissant] Property name must be text. Received: 42
validate_croissant() reports these together with every other issue.
e <- tryCatch(read_croissant(wrong), croissant_read_error = function(e) e)
e$issues@issues[, c("severity", "check", "node")] severity check node
1 error type Croissant
Warnings, such as an undefined term, are signaled as R warnings, and the object is returned. read_croissant() does not apply the structural checks; validate_croissant() applies them and reports read problems as issues of the same table. See Validation.
3.2 Writing
write_croissant(x, path = NULL, pretty = TRUE) writes a Croissant object as JSON-LD. With path = NULL it returns the JSON text as a character scalar of class json; otherwise it writes the file and returns path invisibly.
people <- Croissant(
name = "people",
license = "CC0-1.0",
distribution = list(
FileObject(id = "people.csv", content_url = "people.csv", encoding_format = "text/csv")
)
)
txt <- write_croissant(people)The document carries the standard context of x@version, which croissant_context() returns. The context is omitted from the output below:
doc <- jsonlite::parse_json(txt)
doc[["@context"]] <- "..."
jsonlite::toJSON(doc, auto_unbox = TRUE, pretty = TRUE){
"@context": "...",
"@type": "sc:Dataset",
"name": "people",
"conformsTo": "http://mlcommons.org/croissant/1.1",
"license": "CC0-1.0",
"distribution": [
{
"@type": "cr:FileObject",
"@id": "people.csv",
"contentUrl": "people.csv",
"encodingFormat": "text/csv"
}
]
}
The writer follows these rules:
conformsToalways includeshttp://mlcommons.org/croissant/<x@version>.- Node lists (
distribution,recordSet,field,subField) are written as arrays. A transform list, a reference list, and a string list with one element are written as a single value. contained_inis written ascontainedIn.- Text named by language tag is written as a language map in 1.1 and as an array of
@value/@languageobjects in 1.0. - Each entry of
other_propertiesis written as a key of its node.
birds <- Croissant(name = c(en = "Birds", fr = "Oiseaux"))
jsonlite::parse_json(write_croissant(birds))[["name"]] |> str()List of 2
$ en: chr "Birds"
$ fr: chr "Oiseaux"
birds@version <- "1.0"
jsonlite::parse_json(write_croissant(birds))[["name"]] |> str()List of 2
$ :List of 2
..$ @value : chr "Birds"
..$ @language: chr "en"
$ :List of 2
..$ @value : chr "Oiseaux"
..$ @language: chr "fr"
3.2.1 Round trip
A document read and written again keeps every property: typed properties through the object’s fields, and all others through other_properties. Reading the written document returns an equal object.
path <- tempfile(fileext = ".json")
write_croissant(people, path)
all.equal(read_croissant(path), people)[1] TRUE
3.3 The JSON-LD context
croissant_context(version) returns the standard @context of Croissant "1.0" or "1.1" as a named list.
ctx <- croissant_context("1.1")
length(ctx)[1] 51
ctx[c("cr", "recordSet", "dataType")]$cr
[1] "http://mlcommons.org/croissant/"
$recordSet
[1] "cr:recordSet"
$dataType
$dataType$`@id`
[1] "cr:dataType"
$dataType$`@type`
[1] "@vocab"