review() provides a deterministic assessment of a supervised model. writeup() produces Methods and Results text from recorded model information. ai_review() requests an additional narrative interpretation through rtemis.llm.
2026-10-08 02:54:52 Checking data is ready for training... ✔ [check_supervised]
2026-10-08 02:54:52 ▶ [train]2026-10-08 02:54:53 Training set: 120 cases x 4 features. [summarize_supervised]2026-10-08 02:54:53 Test set: 30 cases x 4 features. [summarize_supervised]2026-10-08 02:54:53 //CPU | serial | 1 worker: algorithm 1 thread, tuning 1, outer resampling 1 [msg_resources]2026-10-08 02:54:53 Shared memory (auto): not sharing training data (6.6 kB) -- this run transfers nothing to workers. [report_shared_memory]2026-10-08 02:54:53 Training CART Classification... [train]2026-10-08 02:54:54 Checking data is ready for training... ✔ [check_supervised]
<SupervisedReview>
CART decision tree (rpart) was used for classification. Balanced accuracy was 0.99 in the training set and 1.00 in the test set.
120 training and 30 test cases; 4 predictors (10.0 minority-class training cases per predictor).
Performance
Training Test Training - test
Balanced Accuracy 0.992 1.000 -0.008
F1 0.992 1.000 -0.008
Accuracy 0.992 1.000 -0.008
Baseline comparisons
Accuracy 1.000 (95% CI 0.884 to 1.000) vs 0.333 (most common training class); McNemar p = 1.9e-06: better
Balanced Accuracy 1.000 (95% CI 0.901 to 1.000) vs 0.333 (chance level): better
Findings
* SINGLE_SPLIT (note): Performance was estimated on a single split, with 30 test cases. The test intervals reflect the number of test cases, not how much the estimate would change with a different split.
Suggestion: Use outer resampling to test on every case and to measure variation between splits; repeat it when cases are few.
* TEST_PRECISION (note): Test balanced accuracy is 1.000 (95% CI 0.901 to 1.000), from 30 test cases.
* BASELINE_ACCURACY (note): Test accuracy of 1.000 (20 cases correct only for the model, 0 only for the baseline; exact McNemar p = 1.9e-06) is better than the 0.333 of always predicting 'setosa', the most common training class.
* BASELINE_BALANCED_ACCURACY (note): Test balanced accuracy of 1.000 (95% CI 0.901 to 1.000) is better than the chance level of 0.333.
Limitations
- Whether the model is useful depends on the question it addresses, the costs of different errors, and how its predictions will be used; performance metrics cannot establish this.
- Preprocessing, decomposition and tuning done by rtemis are fitted on the training cases of each split and only applied to its test cases. Steps taken before the data was passed to rtemis -- selecting or filtering predictors, transforming or removing cases using all the data, or choosing among models by their test performance -- can bias evaluation; this review cannot determine whether that happened.
- Performance estimates apply to cases from the population the data was drawn from; performance on a different population or setting may differ.
The review reports sample sizes, class counts, training and test metrics, baseline comparisons, tuning-range findings, and limitations. For a single split, it computes supported intervals and paired baseline comparisons where the data permit them. Unavailable estimates remain unset.
For resampled models, metrics are summarized across folds. Their variation is descriptive: shared training observations induce dependence between folds. The review does not treat these folds as independent observations for a confidence interval or significance test.
A recorded training/test gap is a diagnostic observation. Model usefulness also depends on the study design, population, and intended decisions. Preprocessing or selection performed before train() may be outside the information available to the review.
31.2 Generate Methods and Results
report <-writeup(mod)report
<SupervisedWriteup>MethodsData
The data comprised 120 training cases and 30 test cases, with 4 predictors.
The outcome, `Species`, had 3 classes: `setosa`, `versicolor` and
`virginica`.
Model
CART decision tree (rpart) was used for classification, fitted with the
rpart [1] package. Table 1 lists the primary hyperparameters of the
algorithm, every hyperparameter that was tuned or specified, and the values
selected during fitting. Supplementary Table S1 lists every hyperparameter
that applied to the fit, its value, how it was chosen and the values tuning
evaluated.
Evaluation
Performance was evaluated on the test set, which was not used for fitting
the model.
Performance measures and statistical analysis
Performance was measured by balanced accuracy, F1 and accuracy. Balanced
accuracy is the mean over classes of the proportion of each class predicted
correctly.
95% confidence intervals of test performance were computed by the
Clopper-Pearson method [2] for accuracy; by a normal approximation with
per-class binomial variances, each computed with one success and one
failure added, for balanced accuracy. Accuracy was compared with always
predicting the most common training class by the exact McNemar test [3].
Balanced accuracy was compared with its chance level (0.333) through its
confidence interval. All tests and intervals are two-sided.
Software
Analyses were performed in R 4.6.1 [4] with the rtemis package 1.4.2 [5]
and rpart 4.1.27 [1].
Table 1. The primary hyperparameters of the algorithm, every hyperparameter
that was tuned or specified, and the values selected during fitting.
Hyperparameter Value
cp 0.01
maxdepth 20
minsplit 2
minbucket 1
prune_cp unset
prune_cp: Unset prunes nothing after the tree is built.
ResultsSample
The training set included 40 `setosa`, 40 `versicolor` and 40 `virginica`
cases, and the test set 10, 10 and 10, respectively.
Performance
Performance on training and test cases is shown in Table 2. On the test
set, balanced accuracy was 1.000 (95% CI 0.901–1.000) and accuracy was
1.000 (95% CI 0.884–1.000). On the training set, balanced accuracy was
0.992.
Comparison with a baseline
Test accuracy (1.000) was higher than that of always predicting the most
common training class (0.333; exact McNemar test, p < 0.001). The 95%
confidence interval of test balanced accuracy (0.901–1.000) lay above its
chance level of 0.333.
Table 2. Performance on training and test cases.
Metric Training Test 95% CI
Balanced Accuracy 0.992 1.000 0.901–1.000
F1 0.992 1.000
Accuracy 0.992 1.000 0.884–1.000
References
[1] Therneau T, Atkinson B (2026). rpart: Recursive Partitioning and
Regression Trees. doi:10.32614/CRAN.package.rpart
https://doi.org/10.32614/CRAN.package.rpart. R package version 4.1.27,
https://CRAN.R-project.org/package=rpart.
[2] Clopper CJ, Pearson ES (1934). The use of confidence or fiducial limits
illustrated in the case of the binomial. Biometrika, 26(4), 404-413.
[3] McNemar Q (1947). Note on the sampling error of the difference between
correlated proportions or percentages. Psychometrika, 12(2), 153-157.
[4] R Core Team (2026). R: A Language and Environment for Statistical
Computing. R Foundation for Statistical Computing, Vienna, Austria.
doi:10.32614/R.manuals https://doi.org/10.32614/R.manuals.
https://www.R-project.org/.
[5] Gennatas ED (2017). Towards Precision Psychiatry: Gray Matter
Development and Cognition in Adolescence. Ph.D. thesis, University of
Pennsylvania.
Supplementary material
Table S1. Every hyperparameter that applied to the fit, its value, how it
was chosen and the values tuning evaluated.
Hyperparameter Value Chosen by
cp 0.01 default
maxdepth 20 default
minsplit 2 default
minbucket 1 default
prune_cp unset
method auto default
model true default
maxcompete 4 default
maxsurrogate 5 default
usesurrogate 2 default
surrogatestyle 0 default
xval 0 default
cost unset
ifw false default
prune_cp: Unset prunes nothing after the tree is built.
cost: Unset uses 1 for every feature.
Not recorded by the model
- The source of the data, the study design, the setting and the eligibility
criteria for cases.
- How the outcome and the predictors were defined and measured.
- Missing data and any processing applied before the data was passed to
rtemis.
- How the test set was selected.
The report links numerical claims to their source fields. It includes software and method references, a main hyperparameter table, a supplementary table of settings and origins, and information the fitted object does not record. include_hyperparameters sets the primary hyperparameters shown in the main table; tuned, supplied, and fit-resolved settings are also included.
2026-10-08 02:54:54 Wrote markdown to /var/folders/cr/lgjpmnv97lg6qtnjt9kph7qw0000gn/T//Rtmpo3tYZi/report-e4ca1cd2ea02/methods-results.md. [write_text]
The generated text supplies the recorded analysis details. Study-specific information, including recruitment, measurement, exclusions, and the intended application, belongs in the complete scientific report.
31.3 Request a language-model review
The following example requires rtemis.llm and an Ollama service with the named model installed; it is not executed when building this guide.
llm <- rtemis.llm::create_Ollama(model_name ="gemma4:e4b")narrative <-ai_review( mod,llm = llm,context ="Teaching example using the iris dataset.")write_text(narrative, "model-review.md")
The result retains the deterministic review, model settings, prompt, and review hash. Statements cite finding codes checked against the review. Those checks establish traceability to the supplied findings; scientific interpretation remains subject to review.
