$$T(n) \approx \tau_0 + n\bar{s}$$

Equation 6 · AI Infrastructure

The Token Tax of Giving a Model More Tools

This equation gives an approximation: it relates the quantities while allowing an approximation.

which, if schemas run at roughly a uniform average size $\bar{s}$ , simplifies to T(n) $\approx$ $\tau_0$ + n$\bar{s}$ . Nothing about this term depends on whether the model calls any tool at all; it is the price of the menu, not the price of the order. And critically, because a stateless inference API resends its full input on every request, T(n) is not paid once at the start of a session — it is paid again, in full, on every single turn for as long as tools remain attached to the request.

Meanings in this article

• $\tau_0$: the tool-independent charge.

Equation guide → · Article →

Equation 17 · AI Infrastructure

The Token Tax of Giving a Model More Tools

This equation gives an approximation: it relates the quantities while allowing an approximation.

Anthropic’s Tool Search Tool is a direct, documented implementation of that idea. The mechanism is described plainly: “You provide all your tool definitions to the API, but mark tools with defe$r_l$oading: true to make them discoverable on-demand,” with a lightweight search tool — built-in options include BM25-based and regex-based search — querying an index of names and descriptions, and the full definition of a matching tool injected into context only once the model actually requests it [ 5 ] . The search tool itself is not free, but its overhead is small and fixed rather than proportional to catalogue size: “Only the Tool Search Tool loaded upfront (~500 tokens)” [ 5 ] . Applied to the same…

Equation guide → · Article →