Extract token log probabilities from a Message or list of Messages

Returns the per-token log probabilities of the assistant's response, when the call was made with logprobs = TRUE. Reading a probability off the first token is better calibrated than asking a model to emit a number, so this is the basis for scoring a binary question.

Usage

logprobs(x, top = FALSE)

Arguments

x

Message object, list of Message objects, or list of lists of Message objects.

top

Logical: If TRUE, add an alternatives list column holding the top-k alternative tokens considered at each position, each as a data.table of token, logprob and prob. The alternatives are only present when the call also set top_logprobs.

Value

For a single Message, a data.table of position, token, logprob and prob, or NULL when the message carries no log probabilities. For a list, a list of those, one per element, so the length of the result matches the length of x.

Details

Supported on Ollama and OpenAI-compatible backends; Anthropic does not return log probabilities, so its messages yield NULL rather than an error.

Author

EDG

Examples

# Requires running Ollama server and gemma4:e4b model
if (FALSE) { # \dontrun{
  llm <- create_Ollama("gemma4:e4b", system_prompt = "Answer Yes or No only.")
  msg <- generate(
    llm, "Is the sky blue?",
    think = FALSE, logprobs = TRUE, top_logprobs = 5L
  )
  logprobs(msg)
} # }