Tagged inference economics · show all articles
A long agent loop resends its own history on every step, pays a fixed tool tax each time, and risks a retry that resends it twice. That arithmetic, not model quality, sets the real ceiling on how far a trajectory can run.
OpenAI, Anthropic, and Google publish input and output prices in the same unit, dollars per million tokens. Three specific ratios genuinely converge across all three rate cards. The token underneath the ratio does not.
A rate card prices an uncached token, a cache write, a cache read, and an output token charged whether or not the answer arrives whole. It says nothing about tool overhead, a retried request, or the safety check every response already passes through.
Retrieval is sold as a cheap fix for a model that does not know your data. Embedding compute, vector storage, ANN queries, reranking, and index freshness are five separate meters running underneath it, and none of them appear in the price of the model call.
A token is a billing unit, not a physical quantity. What a buyer actually pays for is a parallel pass, a serial one, a block of resident memory, and a stretch of reasoning that is charged in full and never returned.