as_reward_fn() adapts a metric, called as metric(prediction, expected),
into a reward function for best_of_n() and refine(), called as
reward(prediction, inputs). The expected value is read from the inputs
under expected_field.
Arguments
- metric
A metric, such as
metric_exact_match(). The metric receivesprediction_fieldof the prediction and the bare expected value, so do not give it afieldof its own.- expected_field
Name of the input that holds the expected value.
- prediction_field
Name of the prediction field to compare. With
NULL, the whole prediction is passed to the metric.
Details
Because the expected value travels with the inputs of run(), the
wrapped module receives it too, and run() warns that it is not declared
in the signature. A module without a custom template writes every input
into its prompt, so it would show the expected answer to the model. Give
the wrapped module a template that leaves that field out, as in the
example. If the input is missing, the reward is 0 with a warning.
See also
Other composition:
best_of_n(),
ensemble(),
module-graph,
pipeline(),
reduce_best_by_metric(),
reduce_first(),
reduce_majority(),
reduce_weighted_vote(),
refine(),
step(),
with_assertions()
Examples
reward <- as_reward_fn(
metric_exact_match(ignore_case = TRUE),
expected_field = "expected",
prediction_field = "answer"
)
reward(list(answer = "paris"), list(question = "Capital?", expected = "Paris"))
#> [1] 1
reward(list(answer = "Lyon"), list(question = "Capital?", expected = "Paris"))
#> [1] 0
if (FALSE) { # \dontrun{
qa <- module(
signature("question -> answer"),
template = "Question: {question}\nReply with a single word."
)
checked <- best_of_n(qa, N = 3L, reward_fn = reward)
run(
checked,
question = "What is the capital of France?",
expected = "Paris",
.llm = ellmer::chat_openai(model = "gpt-6-luna")
)
} # }