Why we build deferred tools in the engine

An agent with three connections has more tools than it can use well. The GitHub MCP server alone offers dozens. Add Sentry and Linear and the model reads several hundred lines of JSON Schema before it reads the question, on every turn.

Two things go wrong. The obvious one is cost: tool definitions sit at the front of the request, ahead of the conversation, and they are resent every turn. The less obvious one is accuracy. A model picks worse as the list grows, and worst between tools that look alike, which is exactly what several connections give it. Every service ships a search and a list.

The constraint that shaped this

There is a clean fix for this in the API layer, and most agents cannot reach it.

Anthropic and OpenAI both ship deferral in the request itself. You mark tools with defer_loading: true, add a search tool, and the model pulls in what it needs. They landed on the same flag name independently, which is a good sign the design is right.

It is also narrow. OpenAI's documentation is blunt about it: "Only gpt-5.4 and later models support tool_search." Anthropic's runs on its own tool types and its own recent models. Google has none of it, and the open request against its Python SDK says so in as many words, that the Gemini API currently lacks this capability, and asks for parity with the other two.

Then there is everything else. Every open model we publish a guide for reaches you through OpenRouter, where there is no defer_loading to set. Not on Kimi K3, not on DeepSeek V4 Flash, not on GLM 5.2.

We are not going to build a feature that only works when you pick the right vendor. So deferral lives in the engine, and the design follows from that one decision.

What everyone else does instead

Deferral without provider support is not new, and the shapes people have settled on are worth naming, because they are the alternatives to what we built. They are ordered here by how much machinery they need.

Tell the user to be careful. This is where most harnesses still are, and it is worth saying plainly rather than skipping to the clever answers. Opencode's documentation warns that MCP servers add to your context and that the GitHub server in particular "can easily exceed the context limit", and the remedy on offer is to enable fewer of them. There is an active push in its issue tracker for a search tool, and the requests are specific about why: the full manifest goes into every turn, and it grows with the ecosystem. Manual management is not a bad answer at three connections. It stops scaling at the point where you cannot predict which tools the job needs.

Retrieve, then bind. The harness indexes every tool, embeds the incoming turn, retrieves the closest few, and puts only those in the request. The model never learns there were others. This is the common framework pattern and it works, but the tools array changes from turn to turn, which matters more than it looks. See the cache section below.

A tool that finds tools. LangChain's langgraph-bigtool gives the agent a retrieval tool over a registry, with semantic similarity over a store you can swap out. The model searches, gets tools back, and calls one. Discovery becomes something the model does rather than something done to it.

Names now, schemas later. Claude Code defers MCP tool definitions by default, loading "only tool names and server instructions" at session start and fetching a schema when Claude commits to a tool. The model always knows the whole surface of what it can do, which is the thing pure retrieval takes away, and it still pays only for the schemas it uses. There is a threshold mode too: set ENABLE_TOOL_SEARCH=auto and it loads schemas upfront while they fit within ten percent of the context window, deferring only the overflow.

It is also the clearest illustration of why we did not build on the provider's version. Claude Code is Anthropic's own harness sitting on Anthropic's own API, and it still gates on the model: tool search needs a model that supports tool_reference blocks, so Sonnet 4.5, Haiku 4.5, Opus 4.5 and later. Point it at a non-first-party base URL, or a deployment that rejects the beta, and it falls back to loading every tool upfront. The vendor's own harness has a fallback path for when the vendor's own feature is unavailable.

Defer the capability, not the tool. Pi skips MCP entirely and deals with the problem one level up. Its skills are capability packages carrying their own instructions and tools, described as "loaded on-demand, progressive disclosure without busting the prompt cache". A skill costs a line until something invokes it. The unit being deferred is a job the agent can do rather than a function it can call, which is a cleaner fit when capabilities are things you author and a worse one when they are a hundred tools somebody else's server happens to expose.

That last phrase is worth dwelling on, because a project that shares none of our design decisions arrived at the same binding constraint. Not the token count. The cache.

We land closest to the tool-that-finds-tools shape, and the difference is where the seam sits. A framework doing retrieval owns your agent loop, so you write your agent its way. A harness built on the provider's feature inherits the provider's model list, fallback path included. Ours is in the engine that was already running the tool call, so there is no extra component, no part of your agent that has to be written a particular way, and no model that turns it off.

Two tools, however many you defer

An agent with any deferred tool gets exactly two more. Not two per connection, and not two per deferred tool. Two.

// tool_search: the tools that match, with their schemas
{ "query": "payroll" }
// call_tool
{ "name": "run_payroll", "arguments": { "month": "2026-07" } }

Each does one thing. A search answers with the name, the description, and the input schema of every match, so one search is the whole distance to a call. There is no fetch-the-schema step in between.

A search gives each tool one name, and call_tool takes that name back exactly as it was given. It is the same name the model would call directly if the tool were not deferred, which means the deferred and non-deferred paths never disagree about what a tool is called.

An empty query matches everything, so a model that does not know what is available starts there rather than guessing at search terms.

The search also covers every tool the agent has, from your worker or from a connection, deferred or not. That gives the answer a property worth having: nothing found means the agent has nothing, not that the model searched the wrong half.

Three places to set it

Deferral is a property of a tool, and each source sets the flag its own way.

[agent.support]
defer_tools = true

defer_tools is the agent's default, and anything stating its own defer overrides it. An agent can mix freely: the tools you want the model to always see stay in the request, beside the two.

The settings have a table form when you need them.

[agent.other.defer_tools]
strategy = "search"
max_matches = 5

max_matches matters more than it looks. A match carries a whole schema, so an answer of many is just the tool list the search was supposed to replace. Five is the default and the answer says how many it left out. It is never zero, because a search that can answer with nothing is a search the model cannot use.

The wrapper stops at the engine

This is the part that makes deferral safe to turn on for an existing agent.

A call_tool becomes the call it names. Not a proxy of it, and not a variant of it: the call, with that tool's own name, its own arguments, its own route, and its own retry policy.

Nothing downstream learns that deferral happened. You can set defer_tools = true on an agent whose worker you have already written and not touch the worker.

Schema validation is the interesting row. The provider never received the definition, so the provider cannot check the model's arguments against it. The engine holds the schema and does that check itself. A related consequence: a deferred tool name has no length limit, because it never reaches a provider to be truncated by one. A call_tool naming a tool the agent cannot reach is refused, and the error names the tools it can.

The request stops changing

The reason to do this in the engine rather than in a framework is the cache.

Tool definitions render before the system prompt, which renders before the conversation. A prefix cache is a prefix match, so anything that rewrites the tool list rewrites the front of the prefix and invalidates every cached token behind it. That is the whole thread.

The common framework approach is to retrieve relevant tools per turn and put those in the request. It saves tokens on definitions and pays for it by re-reading the conversation uncached, which is a bad trade on a long thread and an invisible one in a single-turn benchmark.

Our two definitions are constant. Their names and their text say nothing about which tools exist behind them, so the request does not change when the set behind them does.

The engine decides this from the config alone, never from what a fetch has answered. An agent with defer_tools carries the two tools from its first turn, before it has named a single connection. A connection added in turn fifty moves no definition.

Everything variable lives in the answers instead. An answer is a tool result: at the end of the request, behind the cache, where a change costs nothing.

What it costs

A search is a turn. Sometimes two, if the first query was bad. You are trading latency for a cache that holds and for tool selection the model participates in rather than has done to it.

Against native tool search there is one more hop. Once Anthropic's or OpenAI's model has discovered a tool, the schema is in its request and the call is an ordinary call; ours stays indirect, and the model carries a name across a boundary instead of calling a function it can see. On a weak model that is one more thing to get wrong. If every agent you will ever run is on one provider's newest model, theirs is less indirection for the same outcome and you should use it.

When not to turn it on

If you can name the tools the job needs, do not. Filter instead.

[agent.oncall]
mcp = [{ id = "github", tools = { include = ["*issue*", "*pull_request*"] } }]

A filter costs nothing at runtime, adds no hop, and keeps the request stable. What it costs is reach: the tool set is whatever you decided at config time, and a question needing the tool you filtered out gets an answer built from the ones you kept. That is fine when you know the job. Deferral is for when you do not.

The other alternative is to not have the problem. Split the work across sub-agents, give each a small tool set, and nobody sees two hundred tools because nobody needs them. That is the better answer when the work decomposes and the wrong one when it does not: a triage question spanning Sentry and Linear wants one agent holding both, not a coordinator relaying between two that each hold one.

• Deferred tools: the flags, the two tools, and what does not change.
• Connectors: deferring a whole connection.
• Build a Slack bot on an open model: the models this was built for.