How the Recursive Language Model (RLM) Works

An RLM changes where context lives. A regular predictor serializes its inputs into the model request. An RLM keeps the inputs in an R execution environment and initially shows the model only their names, types, sizes, and previews. The model writes R code to select evidence, observes bounded output, and repeats until it can submit the signature’s typed outputs.

That makes RLM an inference strategy for discovery, not a more elaborate way to answer every prompt.

Decide before you build

Need	Prefer
Short context and a direct answer	`Predict`
Hard reasoning over already selected context	`ChainOfThought`
A known calculation expressed as generated R code	`ProgramOfThought`
External actions or data acquisition through tools	`ReAct` or `CodeAct`
Retrieval from a prepared index	RAG
Adaptive exploration of large or irregular in-memory objects	RLM
Search for a reusable implementation across labeled cases	Flex

Use RLM when the model must discover how to inspect the supplied context. Skip it when a direct R function, SQL query, retrieval rule, or ordinary module already describes the path. Every RLM iteration adds another model call and another code execution.

The release-regression tutorial shows the intended shape: use R to identify an anomalous cohort, inspect one relevant change record, and validate the answer against an independent calculation.

One invocation, step by step

An invocation has six observable stages.

rlm_module() reads the signature and fixes the iteration, recursive-call, and output budgets.
The runner loads the signature inputs into .context. A persistent runner keeps its execution state for the whole invocation.
The outer model receives variable metadata and the trajectory so far, then proposes one R code block.
The runner executes that code. After any stricter runner transport limit, bounded head-and-tail output enters the next model turn; the full source object does not.
A valid SUBMIT(...) ends the loop. An invalid typed submission becomes repairable feedback instead.
If the loop spends max_iterations without a valid submission, a final extraction pass attempts to produce the best typed answer supported by the trajectory; provider or type-validation failure remains terminal.

The core interface is small:

library(dsprrr)

explorer <- rlm_module(
  "document, question -> answer, evidence",
  interpreter_factory = function() {
    r_code_runner(timeout = 30, persistent = TRUE)
  },
  max_iterations = 10,
  max_llm_calls = 6,
  max_output_chars = 10000
)

All signature inputs are available under .context:

nchar(.context$document)
search(.context$document, "renewal|termination")
peek(.context$document, start = 12000, end = 15000)

peek() and search() are conveniences, not a separate query language. The generated program can use ordinary R functions appropriate to the value in .context.

State belongs to one invocation

RLM code is iterative. A value calculated on one turn should remain available on the next, so the runner used for an invocation must provide persistent state.

The recommended lifecycle is a zero-argument factory:

explorer <- rlm_module(
  "records, question -> answer",
  interpreter_factory = function() mcp_repl_runner(),
  max_iterations = 10
)

dsprrr creates one runner from the factory, uses it for that invocation, and shuts it down exactly once on success, error, or interrupt. Factory-backed modules can create isolated runners for concurrent invocations.

A directly supplied runner is caller-owned. dsprrr neither resets nor shuts down it. RLM attempts to remove staged context and invocation-private variables when a call ends. Inspect cleanup warnings and shut down or reset the runner if cleanup fails. No cleanup can undo arbitrary file, option, or network side effects from guest code. Reuse a caller-owned runner only sequentially and within one trust boundary:

local({
  runner <- r_code_runner(timeout = 30, persistent = TRUE)
  on.exit(runner$shutdown(), add = TRUE)
  explorer <- rlm_module(
    "records, question -> answer",
    runner = runner
  )
  # Run explorer sequentially inside this scope.
})

Recursive calls happen through the host

The outer model is responsible for planning and code. Generated code can ask a model to interpret a focused slice:

candidate <- subset(.context$notes, component == "checkout-auth")
interpretation <- llm_query(
  "Which change could reduce successful token refreshes?",
  paste(candidate$note, collapse = "\n")
)
SUBMIT(answer = interpretation)

The replay bridge does not pass model credentials into the recursive call path. Guest access to environment variables and other host resources still depends on the selected runner: only a verified sandbox enforces that boundary. The guest emits a nonce-bound, schema-checked request, dsprrr calls the sub-model in the host process, and the code evaluation is replayed with the response. Ordinary R assignments are committed only after the replay completes, and bridged host tools execute once. Direct file, network, or other external side effects before a query cannot be rolled back and may repeat, so keep pre-query guest work read-only.

Bridge requests use unclassed JSON-compatible values. Missing values, NaN, infinities, and classed R objects are rejected instead of being silently coerced. A ToolDef schema helps the model form a call; the host function must still enforce domain-specific constraints before acting.

sub_lm = NULL inherits the outer model passed to run(). Supply a separate chat when narrow interpretation can use a cheaper model:

explorer <- rlm_module(
  "documents, question -> answer",
  interpreter_factory = function() mcp_repl_runner(),
  sub_lm = ellmer::chat_openai(),
  max_llm_calls = 8
)

llm_query_batched() runs independent focused questions concurrently and returns results in input order. A per-request provider failure becomes an ordered [ERROR] ... slot. An unexpected exception, malformed batch result, or transport-level failure terminates the invocation instead of masquerading as a model answer. Every requested prompt consumes one unit of max_llm_calls. Set that budget from the economics of the task, not from the largest number the provider permits.

Submission is part of the contract

The signature defines both the names and types accepted by SUBMIT():

typed <- rlm_module(
  paste(
    "logs ->",
    "error_count: integer,",
    "severity: enum('low', 'high'),",
    "evidence: array(string)"
  ),
  interpreter_factory = function() mcp_repl_runner()
)

Generated code can submit named values:

SUBMIT(
  error_count = 7L,
  severity = "high",
  evidence = c("AUTH-401 increased", "Refresh retries fell to zero")
)

Missing required fields, extra fields, and incompatible values do not silently become a final answer. Optional fields may be omitted. The validation message joins the trajectory, giving the model a chance to repair its submission. Positional submission is supported, but named fields are easier to audit.

RLM strictly validates explicit ellmer string, number, integer, boolean, enum, array, and object types, including nested combinations. It rejects opaque TypeJsonSchema output nodes at construction because accepting them without a JSON Schema validator would make the repair contract misleading. Express the output with ellmer’s explicit type constructors when using RLM.

If max_iterations is exhausted, fallback extraction reads the trajectory and attempts to return the signature fields. Provider or output-validation failure still terminates the invocation. Treat a successful fallback as degraded completion: it is useful, but it is evidence that the exploration budget or instructions may need work.

Output is bounded evidence

max_output_chars limits how much runner output from each execution enters model history. The default 10,000-character module display retains the head and tail. A runner may impose a stricter limit first: managed MCP rejects file/pager compaction for RLM rather than treating an incomplete preview as evidence. The module bound is not permission to return arbitrarily large control payloads.

Ask for structured output when the trajectory matters:

result <- run(
  explorer,
  records = records,
  question = "Which change preceded the failure spike?",
  .llm = ellmer::chat_openai(),
  .return_format = "structured"
)

result$output
result$metadata$repl_history
result$metadata$iterations
result$metadata$llm_calls
result$metadata$runner_policy

Each trajectory entry records the proposed reasoning, executed code, displayed output, success state, and whether a final submission occurred. Prefer the returned metadata over mutable module state for async or batch work.

One investigation or many

run() always stages each RLM input as one REPL variable. Its R length does not imply a batch: an atomic vector, list, matrix, data frame, or fitted model is one context object for one investigation. This matches the way generated code reads the inputs through .context and avoids silently splitting irregular objects.

Use run_dataset() for multiple investigations. Each data-frame row is one RLM invocation; put rich per-row objects in list-columns. A factory-backed RLM owns one fresh runner per row and can use isolated mirai execution. A caller-owned runner is reused sequentially and rejects concurrent dataset execution.

The counters separate workflow steps from paid provider work. action_calls, recursive_calls, and extraction_calls count logical RLM operations. provider_calls sums the verified provider turns behind them, with component fields for action, recursive, and extraction calls. A cached action or extraction contributes zero provider calls and zero current-run usage. If a backend cannot prove every contributing turn, provider, token, and cost totals remain NA instead of reporting a partial total.

Choose the execution boundary

RLM executes model-generated code. Runner choice is therefore part of the program’s security and data contract.

Runner path	Strength	Limitation
Managed `mcp_repl_runner()`	OS sandbox, network disabled, fresh factory-owned invocation	Workspace writes remain allowed; bounded transport is intended for compact JSON-compatible context
Persistent `r_code_runner()`	Preserves rich R objects in a long-lived callr process	Not a security sandbox; process retains the host user’s permissions
Custom runner	Can provide a remote sandbox or domain-specific resource loading	Must implement and truthfully report the dsprrr runner policy

The one-call rlm() helper chooses a fresh managed MCP runner by default. Install the suggested R package and external executable first:

R -q -e 'install.packages("mcptools")'
uv tool install posit-mcp-repl

Then run a compact investigation:

answer <- rlm(
  "document, question -> answer",
  document = "Owner: team-a\nObligation: rotate signing keys quarterly",
  question = "Which obligations have no owner?",
  .llm = ellmer::chat_openai(),
  .max_iterations = 4L,
  .max_llm_calls = 0L
)

The managed default runs model-generated code under an OS sandbox with network access disabled; it can still mutate allowed workspace files. The final JSON-RPC request must fit the 7 KB wire bound. When the raw request is too large, dsprrr first tries a gzip/base64 wrapper and rejects it before execution only if that request still does not fit. SUBMIT() values, recursive-query context slices, and host-tool arguments also cross a 3,000-byte encoded control-frame boundary. Replayed query and tool results instead count against the next 7 KB raw-or-compressed request. Large data frames and fitted models can use an explicit trusted r_code_runner(persistent = TRUE). Live external-pointer resources generally need a child-side resource loader or a custom runner rather than callr serialization.

The runner boundary is not a privacy guarantee. RLM automatically sends up to 1,000 characters of a structural preview for each input value to the outer model; str()-style previews can include sample values. Anything printed into the trajectory, passed to llm_query(), or returned in the final answer can also be sent to the configured provider. Declared host tools run in the dsprrr host process, outside the guest sandbox, with the host’s permissions.

Action and fallback predictor requests also follow dsprrr’s response-cache configuration, which can include memory and disk. For sensitive investigations, pass .cache = FALSE to run() or configure a memory-only or disabled cache. Recursive llm_query() calls and runner execution are not response-cached.

Repairable and terminal failures

Ordinary R errors are part of exploration. A malformed regular expression or missing column can be shown to the model so it can try a corrected expression.

Interpreter startup, transport, protocol, and shutdown failures are different. They invalidate the execution boundary and terminate the invocation. dsprrr does not disguise those failures as an empty observation or reuse a runner whose state is uncertain.

Three budgets keep the ordinary loop finite:

Argument	Bounds
`max_iterations`	Outer model/code/observation turns before fallback
`max_llm_calls`	Host-side recursive model calls; batch items count separately
`max_output_chars`	Head-and-tail execution output retained per turn

The host bridge also enforces an internal safety ceiling of 1,000 declared tool calls in one generated R step. This is a protocol guard, not a recommended working budget; generated steps should make a small, auditable number of calls.

Start small, inspect the returned trajectory, and increase a budget only when the trace shows useful unfinished work.

Relationship to DSPy

RLMs were introduced by Zhang, Kraska, and Khattab (Zhang et al. 2025). DSPy makes the pattern a module with a per-invocation interpreter, recursive query tools, typed final output, and trajectory metadata. dsprrr follows those execution contracts while using R and ellmer:

inputs appear as .context$name and generated programs are R;
sub_lm = NULL inherits the invocation’s outer ellmer chat;
a factory creates one persistent runner per invocation; a caller-owned persistent runner can be reused sequentially;
invalid typed submissions are repairable observations;
recursive calls and host tools cross a validated replay boundary; and
run(..., .return_format = "structured") returns outputs with trajectory and runner evidence.

The R implementation is not a line-by-line port. In particular, runner and serialization capabilities determine which R objects can cross into an execution environment. Those limits should be visible and testable rather than described as unlimited context.

The practical rule

Use RLM for high-value questions where the useful slice and computation are not known beforehand. Inspect its trajectory as seriously as its answer. When the same exploration pattern repeats, replace discovery with a deterministic R function, a dsprrr pipeline, or an optimized executable program.

For a complete deterministic example, continue to Investigate a Release Regression with an RLM.

References

Zhang, Alex L., Tim Kraska, and Omar Khattab. 2025. Recursive Language Models. https://arxiv.org/abs/2512.24601.

James H. Wade

2026-08-28