Skip to contents

Check cognitive-level consistency across a mapping

Source: R/11_consistency.R
check_level_consistency.Rd

Compares the taxonomy level of every aligned pair: for each edge in a Mapping that survives threshold, the candidate's rank minus the criterion's rank.

Usage

check_level_consistency(mapping, levels, threshold = 0.5, tolerance = 0L)

Arguments

mapping

Mapping object, from map_course_to_sessions().

levels

Classification object, or a list of them, from classify_objectives(). Ids must match the mapping's from_id and to_id.

threshold

Numeric [0, 1]: Minimum strength for an edge to count as aligned.

tolerance

Integer [0, Inf): Rank difference treated as consistent.

Value

data.frame with one row per aligned pair, carrying a run attribute recording taxonomy, threshold, tolerance and the counts behind any warning, plus taxonomy and criteria attributes level_gaps() reads. Pass it to level_gaps() for the per-criterion rollup.

How to read delta

delta is signed and both directions are reported, because they are different findings and neither is universally the error:

  • "under" (delta < 0): the session objective sits below the course objective's cognitive level. A course objective whose every aligned session is under it is the strong finding – nothing in the course asks students to work at the level the course claims to reach. A single low-level session under a high-level course objective is usually scaffolding, not an error, which is why level_gaps() rolls this up per criterion rather than counting flags.

  • "over" (delta > 0): the session objective sits above it. Read this as evidence about the wording of the course objective, not the session: the course is teaching something the outcome statement understates.

Neither is a violation on its own. tolerance widens the band treated as "ok"; adjacent-level disagreement is within the noise of human Bloom coding too, so tolerance = 1 is a defensible setting and is why it is an argument rather than a constant.

Edges below threshold are dropped, not flagged: an unaligned pair has no consistency claim to make. Pairs whose criterion or candidate was never classified, or whose classification failed, are returned with flag = NA and counted in a warning rather than dropped, so a run that silently failed on half its items cannot pass as a clean report.

Author

EDG