6 Planning and Resuming

plan_mapping() counts the work required for a course without calling a model. When supplied with outdir, it reads checkpoints to identify remaining work. The plan does not write checkpoint files.

6.1 Planning a run

library(curriculum)
library(rtemis.llm)

course <- read_course("objectives.yaml", "assessments.yaml")
classification_config <- config_Ollama("your-llm-model", temperature = 0)
scoring_config <- config_OllamaDecision("your-decision-model")

plan <- plan_mapping(
  course,
  classification_config = classification_config,
  scoring_config = scoring_config,
  course_id = "demo",
  scoring_system = "ioc",
  block = "pairwise",
  outdir = "runs/demo"
)
plan
plan@steps
plan@checks
plan@conflicts

Both model configurations are required when inspecting checkpoints. Use the same stage settings and per-call options as the intended mapping run. A basic plan_mapping(course) can count work without model configurations when no checkpoint directory is supplied.

The plan contains input counts, step counts, structural-check findings, checkpoint conflicts, settings, and timing-reference information. Counts with the suffix _left describe work not supplied by compatible checkpoints.

6.1.1 Pairs, evaluations, questions, and calls

The number of item pairs includes repeated text. Deduplication can reduce the number of model evaluations while preserving every original pair in the result. For example, 40 session objectives and 112 assessment items form 4,480 pairs. If the assessments contain 111 distinct texts and the session texts are unique, pairwise scoring requires 4,440 calls per rating pass.

For decision scoring:

Count	Meaning
`evaluations`	Candidate inputs evaluated under the selected grouping
`questions`	Categorical judgments requested across those inputs
`calls`	Planned provider requests, including question chunking

In pairwise mode, all three counts are equal. In joint mode with C criteria and D distinct candidate texts, there are D evaluations, C × D questions, and D × ceiling(C / 64) calls. For an LLM in joint mode, there are D calls per rating pass; decision-question counts are missing.

Console summaries omit evaluation and question counts when both equal the call count. They identify text deduplication when applicable. Planned counts describe the scoring pass and do not include additional attempts caused by failures or subsequent retry passes.

6.1.2 Time estimates

A compatible completed run can supply reference timing:

reference <- readRDS("previous_mapping.rds")
plan <- plan_mapping(
  course,
  classification_config = classification_config,
  scoring_config = scoring_config,
  course_id = "demo",
  outdir = "runs/demo",
  reference = reference
)
plan

The reference must match the relevant model configurations, grouping, scoring definition, and rationale settings. Estimates use recorded seconds per call and the remaining call count. They are approximate: document length, model load, hardware, and request grouping affect elapsed time. Without suitable reference timing, the corresponding estimate is missing. A reference with no recorded timing raises an error. If the reference lacks assessment timing, the plan can borrow its course-to-session rate and identifies that substitution.

6.2 Saving checkpoints

cm <- map_curriculum(
  course,
  classification_config = classification_config,
  scoring_config = scoring_config,
  course_id = "demo",
  scoring_system = "ioc",
  block = "pairwise",
  outdir = "runs/demo"
)

Completed classification is saved as a stage file. Scoring is saved in parts: one completed criterion per rating pass in pairwise mode, or a completed batch of candidates in joint mode. Completed mapping stages and the final combined result are also saved.

Writes use a temporary file followed by a rename. A part becomes reusable after its computation and checkpoint write complete. Classification has no per-item checkpoint in map_curriculum(); an interrupted classification stage is recomputed unless its completed stage file exists.

6.3 Resuming an interrupted run

Rerun the same map_curriculum() call with the same course data, configurations, settings, course_id, and outdir. There is no resume argument. In a new R session, first reload the packages and recreate the course and configurations. Relative paths must resolve to the original directory.

Compatible saved parts are reused, and missing parts are computed. If a run stops after 108 of 111 calls within a pairwise criterion, that incomplete criterion is computed again. Earlier completed criteria are reused.

Checkpoint signatures identify the input text and scoring settings. A conflicting scoring-parts directory stops the mapping call before model work begins; plan@conflicts identifies the differing component. Restore the original settings to resume, or select a different outdir for a new analysis.

Retrying failed scores and resuming interrupted work are separate operations. retry = TRUE, the default, requests another scoring pass for missing alignment scores after the main pass. It does not provide finer checkpoint granularity.

6.4 Progress and timing

Persistent log lines record completed stages and scoring parts. A per-part line reports its call count and elapsed time. Its trailing ETA estimates remaining time in the scoring loop from the mean duration of parts computed during the current call. It is a duration, not a clock time or an estimate for every remaining pipeline stage.

The live console status can display an ETA for the current nested model-call loop. Its scope can therefore differ from the ETA printed beside a completed criterion. Estimates can increase when subsequent parts take longer.

cm@run[["timing"]] records stage timing. Each mapping’s @run[["timing"]] records per-part timing and checkpoint reuse. Saved parts retain their original computation times; elapsed describes the current invocation. See Comparing mappings for the interpretation of these quantities.

6.5 Loading a completed result

For routine reuse, save and load the returned object directly:

saveRDS(cm, "demo_mapping.rds")
cm <- readRDS("demo_mapping.rds")

The final checkpoint also contains the result, wrapped with its signature:

checkpoint <- readRDS("runs/demo/demo_curriculum.rds")
cm <- checkpoint[["value"]]

An interrupted run may have completed scoring parts without a final curriculum file. Resume through map_curriculum() to assemble a complete result from those parts.