Probability of specific tokens at one position — token

Reads the probability of each candidate answer directly off the model's token distribution, rather than parsing a value the model emitted. Asking a binary question and taking P("Yes") at the first position is far better calibrated than asking the model for a number.

Usage

token_probs(x, tokens, position = 1L)

Arguments

x: Message object, list of Message objects, or list of lists of Message objects.
tokens: Character: Candidate tokens to report probabilities for.
position: Integer [1, Inf): Token position to read. Position 1 is the first generated token, which is the answer only when reasoning is off - a thinking model spends its opening tokens on the reasoning channel.

Value

For a single Message, a named numeric vector, one element per token in tokens. For a list, a numeric matrix with one row per element of x and one column per token.

Details

Candidates are matched on trimmed, case-folded token text, and the probabilities of every matching alternative are summed, so "Yes" picks up " Yes" and "yes" too. A candidate that does not appear among the returned alternatives is NA, not zero: its probability is unknown, only bounded above by the smallest one returned. Raise top_logprobs if candidates you care about come back NA.

Author

EDG

Examples

# Requires running Ollama server and gemma4:e4b model
if (FALSE) { # \dontrun{
  llm <- create_Ollama("gemma4:e4b", system_prompt = "Answer Yes or No only.")
  msg <- generate(
    llm, "Is the sky blue?",
    think = FALSE, logprobs = TRUE, top_logprobs = 5L
  )
  token_probs(msg, c("Yes", "No"))
} # }