Tagged retrieval-augmented generation · show all articles
Claude's context window now reaches a million tokens, but independent benchmarks that vary length, position, and lexical overlap show recall degrading well before the window fills — evidence the model's own maker now calls context rot.
Retrieval is sold as a cheap fix for a model that does not know your data. Embedding compute, vector storage, ANN queries, reranking, and index freshness are five separate meters running underneath it, and none of them appear in the price of the model call.
Two axes decide what retrieval-augmented generation becomes by 2035: whether evidence-gathering stays a separate, inspectable step, and whether what it returns can be verified. Crossing them gives four futures, and current evidence does not yet pick one.
A retrieval score and an answer-quality score are counted by different instruments, on different objects, at different points in the pipeline. Either one can look excellent while the other is quietly broken, and most reported RAG numbers do not say which.
Retrieval-augmented generation is usually described as one step: ask, then answer. In production it is six separate stages, each with its own literature, its own failure mode, and its own knob to turn.
Naive, hybrid, graph-augmented, iterative, and agentic retrieval are five distinct engineering systems, not five leaderboard entries. Compared on latency, cost, and failure surface, each wins a different workload — and none of them ranks the others.
Retrieval-augmented generation is usually dated to one 2020 paper. Its lineage is older: 1990s sparse ranking, retrieve-then-read question answering, dense encoders, and now retrieval that decides for itself when to search again.