Read a document to plain text

Deterministic text extraction, dispatched on the file extension. The first stage of extract_assessment(), exported because the text is worth looking at before anything is asked of a model: if this comes back mangled, nothing downstream can recover.

Usage

read_document(file)

Arguments

file

Character: Path to the document.

Value

Character: The document's text, as one string.

Details

Extension	How
.docx, .html, .odt, .rtf, .epub, .tex	pandoc, converted to markdown
.pdf	pdftools::pdf_text(). Requires the pdftools package
.R, .qmd, .Rmd, .md, .txt	Read as lines
.csv	Row cells joined, one row per line

Markup formats go through pandoc to markdown, not to plain text, and that choice matters more than it looks: the list numbering, heading levels and table structure pandoc preserves are exactly what tells extract_assessment() where one item ends and the next begins. Without pandoc, .docx falls back to a tag-stripping reader that loses all of it (with a warning) and the other markup formats are refused.

Source files are read as-is rather than converted: a .R or .qmd file is code, and round-tripping it through pandoc would mangle what the questions are embedded in.

.doc (the pre-2007 binary format) is not supported; convert to .docx.

Author

EDG