The Token Tax of Giving a Model More Tools

Before the model chooses, it has already paid

A tool-using model is usually discussed as though the interesting cost begins the moment it decides to act: it emits a call, something executes, a result comes back. That framing skips the part that happens on every single turn regardless of whether anything is called at all. Before a model can decide not to use a calculator, it has to be told a calculator exists, in enough structural detail that it could use it correctly if it chose to. That telling — the tool’s name, its description, and the JSON Schema of its arguments — is ordinary input text, billed and processed exactly like the rest of the prompt.

A companion piece in this series worked through why a long agentic loop’s retransmitted conversation history grows roughly as the square of step count, and what caching, compaction, and rate limits do about that. This piece stays inside a narrower, prior question. Before any conversation history has accumulated at all — on the very first turn, with an empty transcript — a request already carries a second, independent cost: the tool catalogue itself. How large is that cost as a function of how many tools are attached and how large each one’s schema is? What does a large catalogue do to whether the model picks correctly, as distinct from what it does to the bill? And what, concretely, do the protocols carrying that catalogue — chiefly the Model Context Protocol, MCP — do to keep the catalogue from being paid for in full on every turn regardless of use?

Each of these has a documented answer, from vendor token-accounting pages, from the MCP specification itself, and from a small but consistent body of independent research measuring what happens to tool-selection accuracy as a catalogue grows. None of it requires speculation about model internals. It is closer to reading a utility bill.

What a schema costs before it is ever used

Start with what “the tools parameter” actually contains. Anthropic’s pricing documentation is explicit that the additional tokens from tool use come from three places: the tools parameter itself — names, descriptions, and schemas — the tool_use blocks the model emits, and the tool_result blocks returned to it, and that enabling tools at all triggers a special system prompt the API inserts automatically ^[4]. That system-prompt insertion is a fixed cost, independent of which tools are attached or how many: for Claude Sonnet 5 it is 354 tokens when tool choice is left to the model and 474 tokens when a specific tool is forced, and the documentation is precise that if no tools are provided at all, a tool choice of none uses zero additional system-prompt tokens ^[4]. Every other current Claude model publishes its own figure in the same table — Opus 5 at 286 and 406 tokens, Haiku 4.5 at 496 and 588 — and several built-in tools add a further fixed charge on top: the bash tool adds 244 to 325 tokens depending on model generation, the text editor tool adds 700, and the computer-use beta adds 466 to 499 tokens to the system prompt plus 735 tokens per tool definition ^[4]. OpenAI’s documentation states the same structural fact for its own API in a single sentence: functions are injected into the system message in a syntax the model was trained on, “this means callable function definitions count against the model’s context limit and are billed as input tokens” ^[6].

Call this fixed, tool-independent charge $\tau_0$. On top of it sits the actual content of the catalogue: for $n$ tools attached to a request, each with a schema costing $s_i$ tokens for its name, description, and JSON Schema of parameters, the tool-visibility component of a single request’s input is

which, if schemas run at roughly a uniform average size $\bar{s}$, simplifies to $T(n) \approx \tau_0 + n\bar{s}$. Nothing about this term depends on whether the model calls any tool at all; it is the price of the menu, not the price of the order. And critically, because a stateless inference API resends its full input on every request, $T(n)$ is not paid once at the start of a session — it is paid again, in full, on every single turn for as long as tools remain attached to the request.

Anthropic’s own engineering documentation gives concrete magnitudes for what $n\bar{s}$ looks like in a realistic deployment. A worked example lists five connected MCP servers — GitHub with 35 tools consuming roughly 26,000 tokens, Slack with 11 tools at roughly 21,000, Sentry with 5 tools at roughly 3,000, Grafana with 5 tools at roughly 3,000, and Splunk with 2 tools at roughly 2,000 — for a total the documentation states plainly: “That’s 58 tools consuming approximately 55K tokens before the conversation even starts” ^[5]. A separate figure from the same source is larger still: “At Anthropic, we’ve seen tool definitions consume 134K tokens before optimization” ^[5]. Dividing the first figure through gives a rough sense of scale — roughly 950 tokens of schema per tool on average across that particular set of five servers, an estimate this article derives from Anthropic’s published numbers rather than one the documentation states directly, and one that will vary considerably by tool. A single connected productivity suite can therefore spend more tokens describing what it could do than most single-turn prompts spend on the task itself.

Discovery without a handshake, and its own limits

“Discovery overhead” is often used to mean two different things at once, and MCP’s own recent history is a clean way to separate them. One is the round trip needed to establish that a client and server can talk before anything useful can be requested. The other is the size of the payload once they do.

On the first: MCP’s current specification revision removed the session-establishing handshake that earlier revisions required. The versioning specification states it without qualification — “There is no negotiation handshake. Every request carries its protocol version, and the server accepts or rejects each request independently” — and draws the contrast explicitly: revisions from 2025-11-25 and earlier are termed “legacy” because they “establish a session with an initialize handshake,” while the current, “modern” revision instead conveys protocol version, client identity, and capabilities as metadata on every individual request ^[3]. A server implementing only the modern protocol still exposes an optional server/discover method a client may call up front, but the specification is explicit that this is not required — a client is free to send its first real request inline and handle an UnsupportedProtocolVersionError if the version turns out to be unsupported ^[3]. Removing a mandatory round trip before the first substantive call is a genuine latency reduction, and it is worth naming as a design choice a widely used protocol actually made, not merely a theoretical option.

That leaves the second meaning, and it is where the real cost lives. To discover what a server can do, a client still sends a tools/list request, and that response is the full tool catalogue — the same names, descriptions, and JSON Schemas that later get folded into $T(n)$ on every inference call. The specification requires this operation to support pagination: “Pagination in MCP uses an opaque cursor-based approach, instead of numbered pages,” page size is “determined by the server,” and clients “MUST NOT assume a fixed page size,” with the documentation naming the reason directly — pagination “is especially important when connecting to external services over the internet, but also useful for local integrations to avoid performance issues with large data sets” ^[2]. A very large catalogue therefore costs latency twice over on first discovery: once as however many sequential tools/list round trips it takes to walk the paginated cursor to its end, and again as the token cost of every schema once assembled into a request. The specification also asks servers to “return tools in a deterministic order,” and states why in language aimed squarely at cost rather than correctness: deterministic ordering “enables clients to reliably cache the tool list and improves LLM prompt cache hit rates when tools are included in model context” ^[1]. That is a wire-format requirement written for the economics of the thing consuming it, not for the protocol’s own bookkeeping.

More tools, worse choices: the documented degradation

None of the above would matter much if a larger catalogue were merely more expensive. The harder claim — and the one with the most direct evidence behind it — is that a larger catalogue can also make the model choose worse, independent of cost.

The problem was named early. Gorilla’s authors, building a model specifically to invoke APIs correctly, opened from the observation that despite everything large language models can do, “their potential to effectively use tools via API calls remains unfulfilled,” and found that pairing the model with a document retriever over the API corpus, rather than presenting the full corpus, was what let it adapt reliably to a catalogue of thousands of Hugging Face, TorchHub, and TensorHub APIs while reducing hallucinated calls ^[7]. That is the shape the rest of the evidence fills in with numbers: a raw, undifferentiated catalogue is not the format a model reasons over well, however capable the model.

MCPVerse pushed the test to a genuinely large, real-world scale: more than 550 real, executable tools assembled into an action space exceeding 140,000 tokens, evaluated under conditions of increasing scale. The result is not a uniform collapse, and the paper is careful to say so: “while most models suffer performance degradation when confronted with larger tool sets,” certain agentic models — Claude-4-Sonnet is named specifically — “can effectively leverage expanded exploration spaces to improve accuracy” instead ^[8]. Read plainly, that is evidence both that catalogue size is a real variable with a measurable effect, and that the effect is not identical across systems; a claim that every model degrades past some universal tool count is not what this benchmark shows.

RAG-MCP quantified the fix directly rather than only the symptom. Framing the problem as “prompt bloat and selection complexity” that grows with the number of exposed tools, the authors built an MCP stress test comparing a baseline that places full tool descriptions in context against a retrieval step that selects only the relevant ones first. The baseline scored 13.62% tool-selection accuracy; retrieval-augmented selection reached 43.13%, more than tripling it, while cutting prompt tokens by over half on the same tasks ^[9]. That is a single benchmark’s numbers, not a universal constant, but the direction and the mechanism — filtering before presenting, rather than presenting everything and asking the model to filter internally — recur in the two papers below.

A 2026 study framed the underlying trade-off precisely: show a model too few candidate tools and the correct one may not even appear in the list it sees; show it too many and it struggles to choose among them. Using a chance-corrected metric the authors call Bits-over-Random, they found that on the Berkeley Function-Calling Leaderboard’s 370-tool set, an adaptive shortlisting policy showing only 7 tools on average nearly matched the coverage of a fixed 50-tool list — 90.3% against 90.8% — and, validated directly against Claude Sonnet 4.6, adaptive shorter lists improved selection accuracy to 93.1% against 87.1% for a fixed 5-tool list, a gap that widened to 76.8% against 60.9% on medium-difficulty queries ^[10]. The same paper found a genuine cost on the other side of the trade: on a much larger 3,251-tool registry, a fixed 5-tool shortlist achieved higher overall coverage than one adaptive configuration, because the adaptive policy’s harder queries — where the correct tool ranked 6th to 20th — recovered only 16.7% accuracy on cases a fixed shallow list found zero of at all ^[10]. Shortlisting is not a free win at every scale; it moves the trade-off, it does not abolish it.

A separate, applied study on enterprise agent routing gives a way to decompose exactly what breaks as a real tool catalogue scales, rather than only that something does. Scaling from 10 to 110 candidate agents or tools, the authors report routing F1 on under-specified requests dropping 16 to 23 percentage points across the models tested, and split that drop with an oracle analysis into two distinct components: a retrieval gap, the model’s failure to surface the correct tool at all, and a confusion gap, a roughly 10-percentage-point reduction in the theoretical best-case score that persists even when retrieval is assumed perfect ^[11]. Written as a decomposition of the error a shortlist of size $m$ drawn from a full catalogue of $n$ tools produces, where $\mathrm{tool}^\star$ is the one the request actually calls for and $S_m$ is the shortlist shown to the model:

Widening the shortlist drives the first term toward zero and the second term upward; narrowing it does the reverse. There is no value of $m$ that eliminates both terms at once, which is exactly why the paper reports a design fix rather than a single ideal $m$: embedding-based shortlisting recovered 10 to 11 percentage points of F1 at full scale, consistently across three model families and two providers, and 10 to 17 percentage points on live production traffic, validated against 1,435 human-labeled utterances ^[11]. The fix is not “show fewer tools” as a blanket rule. It is choosing which few to show, per request, well.

Fixing it without shrinking the toolbox: lazy discovery

The most direct answer to a large, expensive, accuracy-degrading catalogue is not to remove capabilities from it. It is to stop loading the whole thing into every request regardless of relevance — the same move RAG-MCP and the shortlisting papers above make at the model-selection level, applied here at the token-accounting level.

Anthropic’s Tool Search Tool is a direct, documented implementation of that idea. The mechanism is described plainly: “You provide all your tool definitions to the API, but mark tools with defer_loading: true to make them discoverable on-demand,” with a lightweight search tool — built-in options include BM25-based and regex-based search — querying an index of names and descriptions, and the full definition of a matching tool injected into context only once the model actually requests it ^[5]. The search tool itself is not free, but its overhead is small and fixed rather than proportional to catalogue size: “Only the Tool Search Tool loaded upfront (~500 tokens)” ^[5]. Applied to the same 58-tool, roughly 55,000-token example described above, that reframes the earlier formula: instead of the full $T(n) \approx \tau_0 + n\bar{s}$, a session that only ever actually needs $m$ of the $n$ available tools pays approximately $\tau_0 + \sigma + m\bar{s}$, where $\sigma$ is the search tool’s own fixed overhead. The fraction of the original tax avoided is

which grows toward its ceiling as $m$ shrinks relative to $n$. Anthropic reports a measured outcome consistent with that shape at their own scale: “Tool Search Tool preserves 191,300 tokens of context compared to 122,800 with Claude’s traditional approach,” described as an “85% reduction in token usage while maintaining access to your full tool library” ^[5]. Against a standard 200,000-token context window, those two preserved-context figures imply roughly 8,700 tokens actually consumed by tool definitions with search enabled against roughly 77,200 tokens without it — an arithmetic inference this article draws from the published figures, not one stated as such in the source. The same documentation reports the mechanism also improved accuracy rather than only cost, on internal MCP evaluations: Opus 4 rose from 49% to 74%, and Opus 4.5 from 79.5% to 88.1%, with Tool Search Tool enabled ^[5].

MCP’s own pagination mechanism is a related, protocol-level version of the same idea, though a coarser one: a client that only ever consumes the first page or two of a tools/list response and stops once it has found what it needs is doing a cruder form of lazy discovery than a relevance-ranked search index, but it is the same principle — do not pull the entire catalogue into a working set before it is needed ^[2]. OpenAI’s function-calling guidance arrives at the identical prescription from the client-application side rather than the protocol side, recommending that if token limits are a problem, developers should try “limiting the number of functions loaded up front, shortening descriptions where possible, or using tool search so deferred tools are loaded only when needed” ^[6] — three separate levers, and lazy loading is explicitly one of them.

Namespacing and grouping: keeping a large toolbox nameable

A second, distinct lever has nothing to do with which tools get loaded and everything to do with what happens when tools from different sources collide. As a catalogue grows past a single server’s own tools — which is precisely what happens once several MCP servers are connected at once, as in the GitHub-plus-Slack-plus-Sentry example above — the chance that two servers each expose a tool called search or create stops being remote.

MCP’s specification anticipates this directly rather than leaving it to convention. Tool names are required to be unique only within a single server, and the specification states the consequence for anything aggregating multiple servers: clients or proxies “MAY encounter naming collisions (for example, two servers each exposing a search tool) and SHOULD implement a disambiguation strategy such as prefixing tool names with a server identifier” ^[1]. It adds a warning worth taking seriously precisely because it is easy to skip: the server’s own self-reported name field “is not guaranteed to be unique across servers and SHOULD NOT be relied upon for disambiguation” ^[1] — the label a server puts on itself is not a safe key to sort by, any more than a component’s own printed part number is a safe substitute for the bin it was actually pulled from.

Namespacing is not primarily a token-cost lever the way lazy loading is — a prefixed tool name costs, if anything, marginally more tokens than an unprefixed one. Its payoff is on the accuracy side of the ledger developed above: a collision between two identically named tools is a pure, avoidable instance of the confusion term in the error decomposition from the previous section, one that no amount of retrieval quality can fix, because retrieval correctly found a tool named search — it simply cannot tell which server’s search was meant without a name that says so. Explicit namespacing is what keeps a large, multi-server toolbox from generating exactly the kind of miscall a well-tuned shortlist would otherwise have prevented.

Schema minimization, and the tension with granularity

A companion analysis in this series argued, from the same MCP specification, that carving one capability into several small, typed tools rather than one undifferentiated tool with a command string is a real design variable with measurable reliability consequences: narrow tools make intent legible to a harness in a way an opaque command string cannot. Nothing in the evidence above overturns that. But it adds a cost axis that argument did not need to weigh, because it was asking a different question — not how many capabilities should exist, but how many schemas should be visible in context at once, and how large each one should be once it is.

OpenAI’s guidance states the token-cost side of schema size plainly, alongside its guidance on tool count: keep “the number of initially available functions small for higher accuracy,” described as a firm recommendation rather than a hard limit — “aim for fewer than 20 functions available at the start of a turn at any one time, though this is just a soft suggestion” — and separately, when token limits bite, “shortening descriptions where possible” is named as one of three available fixes, alongside limiting the count and deferring load ^[6]. That is schema minimization as a distinct lever from lazy loading: a tool that stays loaded because it genuinely is needed every turn can still be made cheaper by trimming its description and its schema’s own verbosity, independent of whether the catalogue around it is large or small.

The two levers compose rather than compete. A large capability surface, split into well-formed, individually typed tools per the granularity argument, can still keep its visible footprint small by deferring the tools a given request does not need — the Tool Search Tool mechanism above — while keeping the schemas of the tools that do stay loaded as lean as their task allows. What does not work is treating “more tools, more thoroughly described” as free simply because a large context window can technically hold the token count. The window holding it and the model reasoning well over all of it at once, on every single turn, are different claims, and only the second one is what the evidence in this article actually measured.

A compact working model

Putting the pieces together gives a short, honest account of where a tool protocol’s overhead actually lives. A single request pays a fixed tool-use tax $\tau_0$ the moment any tool is attached, plus roughly $\bar{s}$ tokens for every one of the $n$ tools visible in that request, repeated on every turn because the underlying inference API carries no memory between calls. Lazy discovery — Anthropic’s Tool Search Tool, or a disciplined use of MCP’s own paginated listing — replaces the $n$ in that formula with a much smaller $m$, the tools actually retrieved, at the cost of a small fixed search overhead $\sigma$. Separately, and not reducible to a token count at all, the choice the model makes over whatever shortlist it is shown carries its own error, decomposable into a retrieval gap that shrinks as the shortlist widens and a confusion gap that grows as it does, with namespacing removing one specific, avoidable source of confusion — a same-named collision — before the shortlisting step ever runs. Schema minimization shrinks $\bar{s}$ directly; it does not touch $n$ or the selection error at all. Four separate levers, four separate places in the same formula, and none of them a substitute for any other.

Predictions, with the observations that would falsify them

These are forecasts, clearly separated from the sourced analysis above. Horizon: 12 August 2028. Assumption throughout: tool-calling remains the dominant interface between models and external systems, and no single vendor’s proprietary format displaces MCP and function-calling-style schemas entirely.

One. Lazy or deferred tool loading — some variant of Anthropic’s defer_loading pattern or MCP-level on-demand retrieval — will become a default-on behavior in mainstream agent frameworks once a connected tool catalogue crosses a moderate size, rather than an opt-in feature a developer must discover. Disconfirmed if the leading agent frameworks at the horizon date still load every connected server’s full tool catalogue into every request by default regardless of catalogue size.

Two. Published tool-selection benchmarks will increasingly report accuracy as a function of catalogue size or shortlist size, rather than a single number at one implicit tool count, because this article’s own sources show that number is close to meaningless without it. Disconfirmed if influential tool-use benchmarks at the horizon date still report a single headline accuracy figure with no stated tool-catalogue size.

Three. Explicit tool namespacing — prefixing by server or source rather than relying on a bare tool name — will become normative practice in multi-server MCP deployments, enforced by client tooling rather than left to server-author discipline. Disconfirmed if widely used MCP clients at the horizon date still resolve same-named tools from different servers with no deterministic, visible disambiguation.

Four. Schema-size accounting — a per-tool token or byte budget surfaced to the developer at authoring time — will appear as a first-class feature in mainstream tool-authoring SDKs, the same way request latency and error rate are surfaced today. Disconfirmed if, by the horizon date, no major agent SDK exposes a per-tool schema-size figure to developers before deployment.

None of these requires a capability discontinuity. Each follows directly from a cost and an error term that are already visible, and already measured, in the documentation and papers cited above.

What to take away

A model pays for its tool catalogue before it decides to use any of it, in tokens that are billed like any other input and resent on every single turn because the interface underneath has no memory of the last one. That payment has a fixed part, published per model and per tool-choice setting, and a variable part proportional to how many tools are attached and how large each one’s schema is — and, separately from cost entirely, a documented accuracy cost that grows with how many of those tools the model has to discriminate among at once, real enough that independent teams working from Berkeley’s function-calling benchmark, an enterprise routing deployment, and a purpose-built MCP stress test all converged on the same qualitative shape: too few candidates and the right tool is not there to find; too many and the model finds it but second-guesses which one it was.

None of the fixes are exotic, and none of them requires giving anything up. Load lazily, so a large catalogue’s fixed and variable tax is paid only for the slice of it a given request actually needs. Shortlist by relevance rather than showing everything, because the retrieval-gap and confusion-gap terms trade against each other and a fixed list size cannot sit at the right point for every catalogue at once. Namespace explicitly, so a collision between two identically named tools is never the reason a well-retrieved shortlist still gets miscalled. And keep the schemas themselves lean, because a smaller $\bar{s}$ is the one lever that helps regardless of how the other three are tuned. Giving a model more tools is not free, and the documentation and the benchmarks now agree closely enough on where the bill actually falls that “just add more tools” is no longer a defensible default — it is a choice with a measured price attached, on both sides of the ledger.