Tagged long context · show all articles
OpenAI, Anthropic, and Google each publish a token count for how much a model can hold. Independent long-context benchmarks measure a different quantity: how much of it the model actually uses correctly, and the two numbers are routinely far apart.
Claude's context window now reaches a million tokens, but independent benchmarks that vary length, position, and lexical overlap show recall degrading well before the window fills — evidence the model's own maker now calls context rot.
Two axes decide what retrieval-augmented generation becomes by 2035: whether evidence-gathering stays a separate, inspectable step, and whether what it returns can be verified. Crossing them gives four futures, and current evidence does not yet pick one.
A retrieval layer does not let a model remember. It submits evidence in support of an answer not yet written. Almost every expensive failure in production RAG follows from confusing the two.