Skip to contents

as_vitals_cost() converts dsprrr cost information into the tibble that a vitals Task's $get_cost() method returns, so costs from both packages can be reported together.

Usage

as_vitals_cost(x, source = "solver", ...)

Arguments

x

One of:

  • a session summary from session_cost() (one row per model);

  • a traces data frame, such as export_traces() output, with model, input_tokens, output_tokens and cost columns (one row per model);

  • a cost summary from get_cost() or an evaluate() result (a single row with the total price, and model and token counts unknown). This currently fails when the total cost is unknown, for example for a model without price data.

source

Label for the source column (default "solver", the vitals convention; vitals also uses "scorer").

...

Unused.

Value

A tibble with columns source, provider (guessed from the model name), model, input and output (token counts), and price (text such as "$0.01").

Examples

traces <- data.frame(
  model = c("gpt-6-luna", "gpt-6-luna", "claude-sonnet-4-5"),
  input_tokens = c(12000L, 8000L, 20000L),
  output_tokens = c(3000L, 2500L, 6000L),
  cost = c(0.21, 0.15, 0.42)
)
as_vitals_cost(traces)
#> # A tibble: 2 × 6
#>   source provider  model             input output price
#>   <chr>  <chr>     <chr>             <int>  <int> <chr>
#> 1 solver OpenAI    gpt-6-luna        20000   5500 $0.36
#> 2 solver Anthropic claude-sonnet-4-5 20000   6000 $0.42

# Nothing has run in this session yet
as_vitals_cost(session_cost())
#> # A tibble: 0 × 6
#> # ℹ 6 variables: source <chr>, provider <chr>, model <chr>, input <int>,
#> #   output <int>, price <chr>

if (FALSE) { # \dontrun{
result <- evaluate(
  classifier,
  testset,
  metric = metric_exact_match(field = "sentiment"),
  .llm = ellmer::chat_openai(model = "gpt-6-luna")
)
as_vitals_cost(result)
} # }