The Economics, Energy, and Physical Limits of Tool Protocols and the Model Context Protocol

A tool list is not free

Ask a model to use a tool and something has to happen first: the model has to be told, in exhaustive and literal detail, what tools exist, what each one accepts, and what each one returns. That telling is not a metaphor for the cost of tool calling — it is the cost, or at least the first and least-discussed part of it. The Model Context Protocol, MCP, standardises this telling: a client asks a server what it offers, the server answers with a typed list, and the model’s context window receives that list before it receives the task the user actually cares about ^[1]. Anthropic’s announcement of the protocol frames it as replacing “fragmented integrations” with “a single protocol” for connecting models to the systems that hold data and expose actions ^[2]. That is a fair description of what MCP solves architecturally. It says nothing about what MCP costs computationally, and the two questions have different answers.

This article is about the second question: the economics, the energy, and the physical limits of giving a model a large tool surface. The argument has three layers, and they compound. First, every tool description injected into context is billed, every turn, whether or not the model ever calls that tool. Second, once a tool is called, its result typically stays in context for the rest of the session, so a session that calls tools repeatedly is not paying a flat tax but a rising one. Third, both of those costs eventually cash out as real physical resources — accelerator memory holding a growing key-value cache, memory bandwidth spent rereading it, and joules dissipated as heat — and those resources have hard ceilings that no amount of clever prompting removes. What follows works through each layer with the arithmetic made explicit, separating what vendors document, what independent research measures, and what is properly this article’s own analysis.

What MCP actually asks a model to carry

Strip away the branding and MCP is a JSON-RPC 2.0 message protocol connecting three roles: a host application that talks to the model, one or more clients inside that host, and one or more servers that expose tools, data resources, and prompt templates ^[1]. When a host connects to a server it can call a tools/list method and receive back a set of tool definitions, each with a name, a natural-language description, and a JSON Schema for its inputs. Nothing in the protocol requires the host to show all of that to the model on every turn — but in practice, almost every current integration does exactly that, because the model has no other channel through which to learn what it can do. The tool definitions get serialised into the same context window as the system prompt and the conversation, and from the model’s point of view a tool description is indistinguishable from any other text it is asked to attend to.

OpenAI’s own function-calling documentation states this plainly for its own tool-calling interface, which is architecturally distinct from MCP but faces an identical accounting problem: function and tool schemas are “billed as input tokens,” and “callable function definitions count against the model’s context limit” — the guidance goes on to recommend limiting how many functions are loaded at once, shortening descriptions, and deferring rarely used tools through a retrieval mechanism rather than declaring them all up front ^[4]. That is a vendor’s own operational advice, offered because the underlying cost is real enough that the vendor has to coach customers around it. Two independent research groups reached the same conclusion from the opposite direction, by trying to give a model thousands of tools rather than a handful. Patil and colleagues built Gorilla specifically to call from a large, frequently changing pool of API specifications, and found it necessary to pair the model with a document retriever rather than have it read the full API catalogue on every call ^[5]. Qin and colleagues went further, assembling ToolBench from more than sixteen thousand real-world APIs and training a dedicated neural retriever whose job is to recommend a short list of relevant APIs for each instruction, precisely because presenting all sixteen thousand descriptions to the model at once is not a viable design ^[6]. Both papers are about capability, not cost accounting, but the engineering choice they converge on — retrieve a few tool descriptions rather than inject all of them — is only necessary because full injection has a price that becomes prohibitive as the catalogue grows.

A useful way to see the shape of the tax is to write out what actually occupies a turn’s context budget:

$T_{\mathrm{system}}$ is the operator’s own instructions, $T_{\mathrm{tools}}$ is the serialised cost of every tool schema currently declared, $T_{\mathrm{exchange}}$ is the live user message and the model’s own reply, and the summation term is every prior tool result the session has decided to keep. The point of writing it this way is that $T_{\mathrm{tools}}$ is paid in full on the very first turn, before a single tool has been called and before it is known whether any tool will be needed at all. A deployment that registers thirty MCP servers, each exposing a handful of tools with a paragraph of description and a nested JSON Schema, can easily spend several thousand tokens of every request on a catalogue the model consults for perhaps one or two entries. That is not a defect in any particular implementation; it is a direct consequence of a protocol whose entire purpose is to describe capability generically, to a reader who cannot be assumed to already know it.

There is a second cost bundled into the first one, and it belongs to a different ledger: description text is not just expensive, it is also unverified. Invariant Labs’ disclosure of MCP tool poisoning attacks demonstrated that a malicious server can embed instructions inside an otherwise ordinary-looking tool description, and that a sufficiently capable model reading that description — not even calling the tool it belongs to — can be steered into leaking data through a completely unrelated, trusted tool, a technique the researchers call a shadowing attack ^[12]. The MCP specification itself acknowledges this class of risk directly, stating that tool descriptions and annotations “should be considered untrusted, unless obtained from a trusted server” ^[1]. This article’s focus is the token and energy bill, not the security posture, but the two are not unrelated: every one of those thousands of tokens of description is also a thousand tokens of content the model reads with no cost-free way to verify it before acting on the trust the protocol extends to it by default.

The session that never forgets

Schema injection is a flat tax, paid once per turn. The larger and less-examined cost is what happens after a tool is actually called. A tool result — a file’s contents, a search response, a database row set — is text, and once it lands in the conversation, most agent harnesses simply leave it there, because removing it risks removing information the model might still need. The consequence is that a session which calls tools ten times across ten turns is not carrying a constant amount of tool-result text forward; it is carrying an amount that grows with every call, and every one of those tokens is reread, in full, on every subsequent turn, because causal-attention models cannot selectively forget part of their own context.

Model it simply. Suppose a session runs for $k$ turns, and each turn adds on average $\Delta$ tokens of new tool result to a transcript that is resent in full on every request, because nothing is evicting or compressing it. The tokens billed on turn $i$ include roughly $i\Delta$ tokens of accumulated result history, so the total billed across the whole session is

Doubling the length of a session does not double the bill; it roughly quadruples it, because the earlier results are being paid for again on every one of the later turns. This is not a hypothetical — it is the direct arithmetic consequence of the same design choice OpenAI documents for its own tool schemas: whatever is placed in the context window is billed as input on every request that includes it, and a growing transcript is a context window that only gets larger ^[4].

The vendor-side mitigation for this is prompt caching, and it is worth stating precisely because it is a real, substantial improvement rather than a full solution. Anthropic’s documentation describes marking a stable prefix of a request — which can include the tool definitions themselves — with a cache control marker, after which subsequent requests that share that identical prefix are charged at roughly one-tenth the price of an ordinary input token for the cached portion, against a write cost of 1.25 times the base price for a five-minute cache and twice the base price for a one-hour cache ^[3]. Applied to a growing conversation, the documented pattern is to move the cache breakpoint forward each turn so that everything up to the previous turn is read from cache and only the newest exchange is freshly billed ^[3]. That is a genuine, vendor-verified reduction in the marginal cost of the pattern described above — call it an order-of-magnitude discount on the repeated portion, not its elimination.

Three things limit how far that discount reaches, and they are properly analysis rather than vendor claim. First, caching requires an identical prefix; the moment a tool result is summarised, reordered, or a new tool is added to the catalogue partway through a session, the cached prefix breaks and the next request pays full price to rebuild it. Second, documented minimum cacheable lengths — several hundred to several thousand tokens depending on the model — mean a session that has not yet accumulated much history gets no benefit at all ^[3]. Third, and most fundamentally, caching reduces the price of rereading; it does not reduce the fact of rereading. The cached tokens still have to be present in the accelerator’s working memory for every forward pass that attends to them, which is where the argument stops being purely economic and becomes physical. Liu and colleagues additionally found that performance itself degrades as relevant information is buried deeper inside a long context — models are measurably less reliable at using information placed in the middle of a long input than information at its start or end ^[7]. A session that keeps every tool result forever is not only paying more; past a certain length, it is also more likely to lose track of the result it actually needs.

From tokens to joules

A billed token and a physical operation are connected by a chain that is easy to state and easy to forget. Self-attention, the mechanism by which a transformer relates every token to every other token in its context, costs compute that scales with the square of sequence length for a single forward pass over that sequence — this has been true of the architecture since it was introduced and is not specific to any vendor. At serving time the more consequential physical resource is not raw arithmetic but memory traffic: generating each output token requires reading the model’s weights and its accumulated key-value cache from accelerator memory, and Pope and colleagues showed that this makes decoding fundamentally memory-bandwidth bound rather than compute bound, with the practical corollary that cost, latency, and achievable batch size trade against one another under whatever sharding and batching strategy a serving system chooses ^[10]. Kwon and colleagues’ PagedAttention work adds the physical detail that matters most here: the key-value cache for every active session occupies real accelerator memory for as long as that session’s context is live, contiguous allocation wastes a large fraction of it, and the number of sessions a given amount of hardware can serve concurrently is bounded by how much of that memory each session’s growing context consumes ^[11]. A large, ungoverned tool surface — many tool schemas kept resident, many tool results never evicted — does not just cost more tokens; it occupies more of a genuinely finite, physical resource for every concurrent user of the system.

Converting billed tokens into energy requires an empirical measurement, not a theoretical one, because the actual joules per token depend on the hardware generation, the batch size, and the power-management policy in use. Samsi and colleagues benchmarked exactly this for LLaMA models running inference across V100 and A100 accelerators, and their results put the energy cost of decoding in the range of roughly three to four joules per generated token for a sixty-five-billion-parameter model, with meaningful efficiency differences between GPU generations and with power-capping able to cut total energy substantially at a modest latency cost ^[9]. Apply that per-token figure to the arithmetic above and the protocol-overhead question stops being abstract: a deployment carrying, say, three thousand tokens of tool schema on every one of a million daily requests is not spending an abstraction, it is spending on the order of several kilowatt-hours a day specifically on describing capability that most of those requests never use, before any of the repeated-result cost from the previous section is even counted.

It is worth being precise about how far that arithmetic can honestly be pushed, because this is exactly the point where an economics-and-energy argument tends to overreach. De Vries’s widely cited commentary in Joule estimated that incorporating a ChatGPT-scale AI system into every Google search would require on the order of twenty-nine terawatt-hours of electricity a year, comparable to the annual consumption of a country such as Ireland, and projected that AI-related electricity demand could grow by roughly eighty-five to one hundred thirty-four terawatt-hours annually by 2027 ^[8]. That figure is a scenario-level projection built on assumptions about adoption and hardware efficiency at a specific moment, not a measurement of what any particular protocol’s overhead contributes, and it should be read as exactly that: a documented, peer-reviewed illustration that inference energy at scale is large enough to matter at a national-grid level, not a number this article can subdivide to say what share of it is tool schemas. What can be said with the sourcing available is narrower and more defensible: the per-token energy cost is measured and nontrivial ^[9], the aggregate scale at which inference now runs is large enough that even small per-request overheads compound into real electricity demand ^[8], and the mechanism connecting the two — tokens that must be read from memory on every forward pass — is architectural, not incidental ^[10].

What actually bounds the tool surface

Put the three costs together and a deployment question comes into focus: given a fixed context window, how many tools can actually be offered before something has to give? A simple ceiling model makes the trade-off explicit. Let $C$ be the model’s usable context window, $T_{\mathrm{system}}$ the fixed cost of operator instructions, and $T_{\mathrm{reserve}}$ a reserved allowance for the user’s message and the model’s own answer. If every tool schema costs on average $\bar{t}_{\mathrm{tool}}$ tokens and all of them are injected in full on every turn, the maximum number of tools a deployment can nominally offer before it must start pruning or retrieving is

The assumption doing the work in that formula — every schema injected in full, on every turn, uncompressed — is exactly the assumption Gorilla and ToolLLM abandoned once their tool catalogues grew past a few dozen entries, replacing blanket injection with retrieval that surfaces a handful of relevant tool descriptions per request instead of the whole catalogue ^{[5, 6]}. That substitution changes $\bar{t}_{\mathrm{tool}}$ from “every tool’s cost, always” to “a few tools’ cost, most of the time,” which is the only lever in the equation that scales sublinearly as the number of available tools grows. It is also, notably, a departure from what MCP’s own discovery model assumes by default: tools/list is designed to hand back the full capability set a server offers, and nothing in the base protocol specifies how a host should decide to show only part of that set to the model ^[1].

Retrieval buys back context budget, but it does not touch the physical ceiling underneath it. That ceiling is set by accelerator memory, not by the protocol: every concurrently active session’s context — schemas plus accumulated results — has to live somewhere in HBM as key-value cache while that session is being served, and Kwon and colleagues’ central finding is that this memory, not compute, is usually what limits how many sessions a given cluster of accelerators can serve at once ^[11]. A deployment can shrink what it injects per session through retrieval and caching, but the number of long-context, tool-heavy sessions a fixed amount of hardware can hold open simultaneously is bounded by a fixed amount of physical memory, full stop. Adding tools, adding tool servers, or extending how long a session is allowed to run before its context resets are all, underneath the product decision, decisions about how that finite memory gets allocated.

Mitigations, and the costs they trade in for

None of the levers available here are free; each converts one cost into a different one, and a deployment has to choose which cost it would rather carry.

Prompt caching converts a recurring compute cost into a recurring memory-residency cost: the documented ten-times discount on cached reads is real, but it depends on the cached prefix staying resident and unchanged, which means holding it in place rather than reclaiming that capacity for another session ^[3]. Retrieval-based tool selection, in the manner Gorilla and ToolLLM demonstrate, converts a token cost into an engineering and latency cost: an extra retrieval step has to run before the model ever sees a schema, and it has to be right often enough that the model is not left unable to find a tool it actually needed ^{[5, 6]}. Aggressively evicting or summarising old tool results converts a token cost into an accuracy risk, since the summarised version may drop exactly the detail a later turn needed — the same failure mode Liu and colleagues document even without summarisation, simply from information sitting deep inside a long, unmodified context ^[7]. And treating every tool description as untrusted input, which the MCP specification itself recommends, converts an economic question into a verification cost: describing capability cheaply and verifying that the description is honest are two different jobs, and Invariant Labs’ shadowing-attack findings show that skipping the second one has consequences that have nothing to do with the token bill ^{[12, 1]}.

At the infrastructure level, the same trade shows up as a step function rather than a smooth curve. Memory capacity does not expand by the token; it expands by the accelerator, the node, or the rack, each addition arriving with its own fixed power draw and its own fixed capital cost regardless of whether the extra headroom it provides is fully used from day one. A deployment deciding whether it can afford to register one more MCP server, or keep sessions open twenty percent longer, is in practice deciding whether it is ready to cross into the next such increment — and the honest way to answer that is to have already done the token-to-memory-to-energy arithmetic above, rather than discovering the ceiling when a cluster that was serving comfortably starts rejecting new sessions.

Predictions, with the observations that would falsify them

These are forecasts, explicitly separated from the sourced analysis above. Horizon: 22 August 2029.

One. Tool-selection retrieval — showing a model a filtered subset of a large registered catalogue rather than the whole thing — will become a default feature of mainstream agent harnesses rather than a specialised research technique. Disconfirmed if leading agent frameworks in 2029 still inject full tool catalogues by default for deployments with more than roughly fifty registered tools.

Two. Providers will begin publishing per-session or per-tool-call token-overhead figures as a standard part of usage dashboards, because the caching and retrieval mitigations above only work if operators can see where the tokens are going. Disconfirmed if mainstream API usage dashboards in 2029 still report only aggregate input and output token totals with no attribution to tool schemas or tool results specifically.

Three. Reported instances of tool-description-based prompt injection, in the pattern Invariant Labs documented, will continue to be found in new tool ecosystems as they scale, because the underlying incentive — an untrusted description read automatically and by default — is structural to the protocol rather than a bug in any one implementation. Disconfirmed if a widely adopted technical fix (such as mandatory signed, non-executable tool descriptions) removes this class of report from major security disclosures by 2029.

Four. The gap between the context a large tool catalogue could in principle occupy and the context deployments actually spend on it will keep widening, driven by retrieval and caching rather than by context windows growing faster than tool catalogues do. Disconfirmed if growth in advertised context window sizes outpaces growth in registered tool catalogue sizes across major deployments, making raw injection viable again without retrieval.

None of these requires a new capability breakthrough. They follow from the structure already documented here: a protocol that must describe capability before it is known to be needed, a serving system whose cost is bound by memory traffic rather than cleverness, and an accelerator whose finite HBM is the actual, physical last word on how much of that description a cluster can afford to hold open at once.

What to take away

A tool list is an invoice presented before any work is agreed to. MCP and protocols like it make that invoice standard and portable, which is a genuine, documented improvement over bespoke integrations ^{[2, 1]} — but standardising the description does nothing to shrink it, and every one of its tokens is read, billed, and eventually paid for in joules on every turn that includes it. Repeated tool-result re-reading turns a flat tax into a compounding one unless something actively intervenes, and the strongest available intervention, prompt caching, discounts the recurring cost without removing the recurring residency it depends on ^[3]. Underneath both costs sits a physical ceiling that no pricing change moves: key-value cache is memory, memory is finite per accelerator, and the number of long, tool-heavy sessions a cluster can hold open at once is set by that finite quantity before it is set by anything a product team decides ^{[11, 10]}.

Treat a large tool surface as a capacity-planning problem, not a prompting problem. Ask what the catalogue costs before any tool is called, what a session costs by its last turn rather than its first, and what physical memory backs the sessions actually running concurrently — and if any of those three numbers is missing, the tool surface in front of you has not yet been costed, only built.