Map session objectives to assessment items — map_sessions_to

Scores every (session objective, assessment item) pair. The other direction of the map: map_course_to_sessions() asks whether a session serves an objective; this asks whether an assessment item measures one.

Usage

map_sessions_to_assessments(
  session_objectives,
  assessments,
  course_id,
  llmconfig,
  scoring_system = "ioc",
  block = c("pairwise", "joint"),
  repeats = 1L,
  rationale = TRUE,
  verbosity = 1L,
  outdir = NULL,
  ...
)

Arguments

session_objectives: List of character vectors: Session objectives, named by session.
assessments: List of character vectors: Assessment items, named by assessment. Names become part of the ids, so "Midterm" and "Final" stay distinguishable in the long form.
course_id: Character: Course identifier, used to build stable ids.
llmconfig: rtemis.llm::LLMConfig: Model configuration, from config_Ollama(), config_OpenAI() or config_Anthropic(). One object carries the model, temperature, endpoint and key handling, so an OpenAI-compatible gateway needs no special support here – see the OpenRouter example. Only the model name, backend, base URL and temperature are recorded in run; an API key held on the config is never persisted.
scoring_system: Character or ScoringSystem: Scoring system. See list_scoring_systems().
block: Character {"pairwise", "joint"}: Prompt block structure.
repeats: Integer [1, Inf): Independent raters per pair. IOC and CVI are panel instruments and are defined over several raters; with repeats = 1 they still work but their thresholds do not mean what they were calibrated to mean.
rationale: Logical: Whether to ask for a one-sentence justification per cell. Rationales are what make a mapping reviewable, so this defaults to TRUE – but they dominate generation time under constrained decoding, since the free-text field is far longer than the label it explains. Set FALSE for a fast pass when only the matrix is wanted.
verbosity: Integer [0, Inf): Verbosity level.
outdir: Character: Directory to write resumable parts to, or NULL for a run that keeps nothing. Each criterion's scores are written as they are produced, and a later run over the same objectives with the same settings reads them back instead of scoring them again.
...: Additional per-call arguments forwarded to rtemis.llm::generate(), e.g. temperature, seed.

Value

Mapping object.

Details

The engine is shared – only the relation differs, which changes how the prompt is worded and which scoring systems warn. Prefer a scoring system whose recommended_for includes "session_to_assessment"; list_scoring_systems() shows which. cvi is the closest to the published instrument for this direction, since content validity was defined over exactly this judgement.

Reading the result

The criterion is the session objective and the candidate is the assessment item, so in check_level_consistency() a negative delta means the item is below the level of the objective it measures – the course taught analysis and the exam asked for recall. That is the classic assessment misalignment, and it is the one direction where "under" is nearly always a defect rather than scaffolding.

Author

EDG

Examples

if (FALSE) { # \dontrun{
m <- map_sessions_to_assessments(
  obj[["sessions"]],
  obj[["assessments"]],
  course_id = "datasci213",
  llmconfig = config_Ollama("gemma4:e4b", temperature = 0, think = FALSE),
  scoring_system = "cvi"
)
} # }