addition

The passage around this formula

Call this fixed, tool-independent charge $\tau_0$ . On top of it sits the actual content of the catalogue: for n tools attached to a request, each with a schema costing $s_i$ tokens for its name, description, and JSON Schema of parameters, the tool-visibility component of a single request’s input is $T(n) = \tau_0 + \sum_{i=1}^{n} s_i$. which, if schemas run at roughly a uniform average size $\bar{s}$ , simplifies to T(n) $\approx$ $\tau_0$ + n$\bar{s}$ . Nothing about this term depends on whether the model calls any tool at all; it is the price of the menu, not the price of the order. And critically, because a stateless inference API resends its full input on every request, T(n) is not paid once at the start of a…

Read this part in the article →