RAG in 2035: Four Scenarios, Their Signals, and What Would Falsify Them

Forecasting a function, not an acronym

Nine years is long enough for a three-letter acronym to stop meaning anything precise. By 2035, “RAG” may denote a legacy pattern maintained out of habit, a compliance requirement written into a procurement contract, a term nobody says anymore because the behavior it named has fused into ordinary model operation, or something close to what it means today. The useful forecasting object is therefore not the acronym but the function retrieval currently performs: selecting a bounded, inspectable body of evidence from a larger corpus and conditioning an answer on that evidence rather than on the whole corpus at once.

A forecast about that function has to keep four registers separate, because a single sentence about retrieval’s future reads very differently depending on which one it belongs to.

• Fact. A measured result tied to a specific study, model, dataset, and date.
• Vendor claim. A capability or benefit reported by the organization that built the product — informative, not independent.
• Analysis. A consequence derived from a stated mechanism, such as a cost model or an incentive structure.
• Prediction. A statement about the future capable of being wrong, paired with the observation that would show it wrong.

This article uses all four, labeled as it goes, and it deliberately does not pick a favorite among the four scenarios it builds. That refusal is not a hedge. It is the honest output of a method that crosses two genuinely uncertain axes and reports what each of the resulting four cells implies, rather than reading the present as evidence for whichever future the author finds congenial.

The documented present

Fact. Retrieval-augmented generation was defined, not as memory, but as a marginalization over retrieved evidence. Lewis and colleagues introduced RAG as a combination of a pretrained parametric model and a non-parametric dense vector index queried by a neural retriever, explicitly distinguishing the two kinds of store rather than treating retrieval as an extension of what the weights already hold ^[1]. Everything below is a forecast about what happens to that non-parametric half — whether it stays a separately identifiable component or is absorbed into something else.

Fact. Context windows have grown far faster than retrieval has become unnecessary. Google’s own developer documentation for its Gemini models states plainly that many of them now ship with context windows of one million tokens or more, and it addresses the RAG-versus-long-context question directly: the documentation argues that a sufficiently large window “invites a more direct approach” of placing relevant material in the prompt rather than retrieving it, while also cautioning that needle-in-a-haystack accuracy on a single fact does not transfer cleanly to scenes with multiple facts to find at once ^[8]. That is a vendor’s own product documentation making a claim in favor of its own capability, and it should be read as exactly that — informative about product direction, not an independent benchmark result.

Fact. Independent measurement complicates the vendor framing in two specific ways. RULER extended needle-retrieval evaluation into multi-hop tracing and aggregation tasks and found that although the evaluated models all claimed context sizes of 32,000 tokens or more, only about half of them maintained satisfactory performance at that length, with substantial further degradation as length increased ^[2]. A context window is a capacity claim; effective length is a separate, empirically smaller quantity, and the gap between the two is exactly the space in which retrieval continues to do useful work regardless of how large windows get.

Fact. Even when the right information is present in a long input, position changes whether it gets used. Liu and colleagues varied where the relevant passage sat within a long context and found performance highest when it appeared at the very beginning or the very end, degrading significantly when the model had to draw on material in the middle — a pattern that held even in models built specifically for long contexts ^[3]. Pouring a corpus into a window does not, by itself, guarantee that every part of it is used the same way; the window has interior geography, and that geography does not disappear as the window grows.

Fact. The retrieval-versus-long-context question has also been measured directly, on cost as well as accuracy. Li and colleagues compared both approaches on public benchmarks and found that when a model was resourced sufficiently, long-context prompting consistently outperformed retrieval on average accuracy — while retrieval remained substantially cheaper. Their proposed Self-Route method routes each query to whichever approach the model believes it can answer with, matching long-context accuracy at a fraction of the cost ^[4]. Read precisely, this says retrieval is not obsolete; it says that on corpora small enough to fit in a window, spending the tokens buys some accuracy, and that spending is worth avoiding when it can be routed around.

Fact. Two vendors have already shipped products that treat pieces of this problem as worth standardizing rather than leaving bespoke. Anthropic’s Citations feature breaks source documents into sentence-level chunks at ingestion and has the model attach citations linking specific output claims back to those chunks as part of generation itself, rather than as a post-hoc annotation pass, with the vendor reporting — as its own internal evaluation, not an independent benchmark — up to a 15% recall-accuracy improvement over custom citation implementations ^[7]. Separately, Anthropic’s Model Context Protocol defines an open specification for how a model-facing application connects to external data sources and tools, replacing bespoke per-source integrations with one protocol that both data providers and model clients can implement against ^[6]. Neither product is a standard for evidence provenance specifically. Both are evidence that vendors see value in making some part of the fetch-and-cite pipeline less bespoke.

Fact. A generative-AI risk-management standard already treats provenance as a named concern, independent of any RAG-specific requirement. NIST’s Generative AI Profile, a companion to the AI Risk Management Framework, sets out voluntary practices for organizations building and deploying generative systems and lists tracking data and content provenance among its risk-mitigation recommendations ^[10]. This is broader than retrieval — it addresses training data and generated content as well as retrieved evidence — but it establishes that provenance tracking already has an institutional home outside any individual vendor’s product.

Fact, current adoption. Enterprise practice is unsettled in ways that are directly measurable right now. Reporting on enterprise retrieval deployments in early 2026 found that intent to adopt hybrid retrieval — combining dense and sparse methods rather than relying on embeddings alone — roughly tripled quarter over quarter, even as a substantial share of surveyed organizations reported having no production RAG system running at all; the same reporting described the standalone vector-database category losing adoption share to custom in-house stacks and to retrieval features built directly into model providers’ own platforms ^[11]. Separately, an industry analysis of the vector-database market in mid-2026 argued explicitly against a consolidation narrative, describing the market as fragmenting by workload scale and deployment environment across several actively competing products rather than collapsing toward a single winner, and noting a specific incumbent’s reported revenue decline over the same period ^[12]. That analysis was published by a company that sells a competing database product, and its framing should be read with that interest in mind — but the underlying descriptive claim, that no consolidation has yet occurred among standalone vector-index vendors, is consistent with the adoption data in the paragraph above it.

Fact, agentic direction. The most active area of applied RAG research in 2025 and 2026 has been agentic: retrieval systems in which an autonomous loop decides what to fetch, evaluates what it got, and decides whether to fetch again, rather than executing one fixed retrieve-then-generate pass. A survey of this literature characterizes standard RAG as constrained by static workflows unsuited to multi-step reasoning, and describes agentic RAG as embedding planning, reflection, tool use, and multi-agent collaboration directly into the retrieval process so that strategy can change mid-task ^[9]. That survey aggregates a fast-moving, largely non-peer-reviewed literature, and its own characterization of “transformative” progress should be read as a claim advanced by the survey’s authors rather than as an established finding.

Two axes, and why they are only partly independent

Two variables organize what happens to retrieval between now and 2035, and both are consequential and genuinely open.

Axis A — locus. At one end, evidence-gathering remains an explicit, externally inspectable step: a distinct index, a distinct query, a distinct ranked list a person or an auditor can point to before generation happens. At the other end, it dissolves into the model’s own context handling — a continuous blend of parametric knowledge, whatever was pasted into a huge window, and whatever an agent’s tool calls happened to pull in along the way, with no single retrieval stage left to name.

Axis B — verifiability. At one end, the span behind a claim carries persistent, checkable provenance: which source, which version, which query produced it, and permission to have been used at all. At the other end, evidence is an unverified assertion — the generator’s bare claim that something supports a statement, with no chain back to a specific fetched object a skeptical reader could pull and check.

The two axes are related but not reducible to one another, and the coupling has an economic root worth making explicit. Autoregressive decoding is bounded by how much has to be read from memory to produce each token — the weights, plus whatever context has accumulated — and Pope and colleagues formalized exactly this partitioning of cost between arithmetic and memory movement in transformer serving ^[5]. Write $n$ for the number of tokens a system must hold in an answer’s working set. Pasting a whole corpus of size $M$ into context puts a floor under $n$ near $M$ itself; retrieving costs roughly the price of an index lookup plus the price of reasoning over the $k \ll M$ tokens actually returned:

As a corpus grows, the two costs diverge regardless of how cheap or how large any single model’s window becomes, because $M$ keeps growing too. This is the economic argument for why explicit retrieval persists even in scenarios where verifiability is weak: corpus growth, not just model quality, keeps a bounded fetch cheaper than a full paste. It says nothing about whether that fetch is checkable, which is exactly why Axis B has to be argued separately rather than assumed to follow from Axis A.

Four scenarios

Scenario 1 — The Evidence Layer (explicit, verifiable)

Condition. Retrieval remains a distinct, inspectable step, and the evidence it returns carries provenance that survives into the output: source, version, query, and permission, checkable by something other than the generator’s own word.

Mechanism. Citation-grounded generation of the kind Anthropic already ships ^[7], protocol-level standardization of how models reach external sources of the kind MCP already provides for tool and data access ^[6], and provenance-tracking practice of the kind NIST already names as a risk-management recommendation ^[10] converge into a layer that sits between corpora and generators as ordinary infrastructure. Retrieval survives as a named component precisely because naming it is what makes it auditable. Regulated and consequential domains adopt it first, because they already have to answer “what was this based on” for reasons unrelated to model capability.

Horizon. Visible in narrow, high-stakes domains — regulated finance, clinical documentation, legal research — by 2030; broad enough to be an unremarkable default for general-purpose assistants only by 2035, if at all.

Assumptions. Citation and provenance features remain cheap enough in latency and token cost that vendors keep shipping them by default rather than as a paid add-on; standardization efforts like MCP extend from tool and data access to evidence provenance specifically, which is a materially harder problem they do not yet solve; and demand for auditability continues to come from regulation and incident response rather than from voluntary adoption alone.

Observable indicators. (1) Citation or provenance features shipping as default-on rather than opt-in across at least two major model providers. (2) A published interoperability effort — extending MCP or a comparable protocol, or a new specification — that standardizes not just how a model reaches a source but how it records having used one, in a form a third party can check without the original vendor’s tooling. (3) Regulatory guidance or procurement requirements in at least one regulated sector explicitly requiring retrieval provenance as a condition of deployment.

Disconfirmation. This scenario fails if, by 2032, no cross-vendor provenance standard exists even in draft form, and the leading providers’ own citation features remain closed, proprietary formats that cannot be checked or exchanged outside each vendor’s own tooling.

Scenario 2 — The Silo Economy (explicit, unverifiable)

Condition. Retrieval remains a distinct step — there is still a recognizable index, a recognizable query, a recognizable ranked list — but no persistent, checkable provenance travels with what gets returned. Each vendor’s stack, ranking function, and citation format stay proprietary and mutually unintelligible.

Mechanism. This is close to where 2026 already sits, and the present evidence argues it could simply persist rather than resolve. The vector-database landscape is not consolidating around a small number of independent index vendors, as one might have guessed a few years ago; instead, reporting from mid-2026 describes standalone products losing adoption share specifically to two different absorbers — custom in-house retrieval stacks on one side, and retrieval features built directly into model providers’ own platforms on the other ^{[11, 12]}. If that pattern continues, retrieval consolidates not around a small set of independent vector-index companies but around a small number of large model providers, each running retrieval as a closed feature of its own platform. The step stays explicit — you can point to “the retrieval call” in the logs — but nothing about it is portable, comparable, or auditable across providers, because each one owns its own silo end to end.

Horizon. This is closer to a description of present dynamics extending forward than a distant scenario; a meaningful test is available by 2029, once at least one full model-generation cycle has passed under current adoption patterns.

Assumptions. Provider-native retrieval keeps improving fast enough that switching to a bespoke or interoperable stack stops looking worth the engineering cost for most teams; no external pressure — regulatory, competitive, or reputational — forces citation-format interoperability; and enterprises continue to prioritize retrieval accuracy and integration convenience over auditability when the two trade off.

Observable indicators. (1) The count of enterprises running retrieval as a bespoke or third-party layer, rather than a model provider’s built-in feature, continuing to decline, extending the shift already visible in 2026 reporting. (2) No citation or provenance format achieving adoption across more than one major provider’s platform. (3) Migration cost between providers’ retrieval stacks, measured in re-indexing and re-tuning effort, remaining high or rising.

Disconfirmation. Falsified if, before 2030, a citation or provenance format is adopted by two or more major model providers in a form that lets a claim’s evidence be checked without vendor-specific tooling — that would mean the silo boundary had been breached regardless of how consolidated the underlying compute remains.

Scenario 3 — Agentic Custody (implicit, verifiable)

Condition. There is no longer one discrete “retrieval stage” a diagram would show as a box between a query and a generator. Instead, an agent decides what to fetch, when, and whether to fetch again, as an ordinary part of a longer tool-using loop — but each individual fetch remains logged and attestable, so provenance survives even though the pipeline shape does not.

Mechanism. The direction of applied research already points here. Agentic RAG replaces a single fixed retrieve-then-generate pass with a loop of planning, retrieval, reflection, and re-retrieval, adapting strategy to the task rather than following one predetermined path ^[9]. What keeps this cell distinct from Scenario 4 is that the individual tool calls inside that loop are exactly the kind of thing a protocol like MCP already makes loggable: each fetch is a discrete, typed request to a named external source, whether or not anyone still calls the aggregate behavior “a retrieval pipeline” ^[6]. The auditable unit shifts from “the retrieval step” to “each tool call the agent made,” which is a different bookkeeping problem, not an absence of bookkeeping.

Horizon. Emerging in coding and research agents by 2028, where tool-call logging is already close to a debugging necessity; general enough to apply across consumer and enterprise assistants only by 2035.

Assumptions. Tool-calling protocols retain or extend structured, inspectable call records as they add capability, rather than trading logging away for speed; agent frameworks treat call-level attestation as a default rather than an optional feature developers must opt into; and the cost of logging every call does not become prohibitive as agents make many more, smaller calls than a single-pass retriever would have made.

Observable indicators. (1) Agent frameworks shipping structured, queryable logs of tool and retrieval calls as a default rather than a debug-only feature. (2) Published incident post-mortems for agentic systems that successfully reconstruct which fetched source produced a specific wrong claim — evidence that the granular record is actually usable after the fact, not merely generated and discarded. (3) Evaluation suites for agentic RAG that score attribution per tool call rather than only end-to-end task success.

Disconfirmation. Falsified if, by 2031, agentic systems in wide deployment cannot reconstruct after an incident which specific fetch produced a disputed claim — meaning the granularity of tool calls exists at execution time but the record does not survive or is not queryable, which would collapse this scenario into Scenario 4 in practice regardless of the underlying architecture.

Scenario 4 — The Wide Window (implicit, unverifiable)

Condition. Both the distinct pipeline stage and the checkable provenance chain disappear. Context windows and agent memory grow large and cheap enough that engineers simply pour candidate material in — parametric knowledge, pasted documents, prior tool outputs, other agents’ summaries — without a persistent record of which parts of that blend actually supported which output claim.

Mechanism. This is the scenario in which retrieval “loses its distinct identity” in the plainest sense: not because the underlying problem of selecting relevant evidence goes away, but because nobody builds or preserves the bookkeeping that would let a claim be traced back to a specific source. Two documented facts make this more than idle speculation. Vendors already argue directly that sufficiently large windows invite pasting material in rather than retrieving it selectively ^[8], and doing so is measurably imperfect in exactly the way that matters for evidentiary claims: relevant material lodged in the middle of a long input is used less reliably than material at the edges, a positional effect observed even in models purpose-built for long context ^[3]. Put the two together and the risk is specific — not that long-context systems fail outright, but that they fail exactly like the “memory” framing this pillar has argued retrieval was never supposed to resemble: confident answers with no legible account of where they came from, and degraded use of the material that formally is present but happens to sit in the wrong place.

Horizon. Already visible in ungoverned consumer and prototype use as of 2026; the open question is whether it remains confined there or becomes the default in production systems by 2032–2035.

Assumptions. Effective context length and cost continue improving fast enough that engineering teams find it easier to paste more in than to build and maintain a separate retrieval and provenance layer; no incident or regulatory pressure forces the bookkeeping back in; and the accuracy gains long-context prompting has already demonstrated on benchmarks where it is well resourced ^[4] generalize to production settings without a compensating investment in traceability.

Observable indicators. (1) Production systems handling permissioned, continuously updated, or regulated corpora moving predominantly to whole-corpus or whole-history prompting rather than governed retrieval. (2) Rising rates of unattributable claims in audits of long-context system outputs, where a stated fact cannot be traced to any specific input span. (3) Continued vendor guidance recommending context-stuffing over retrieval for the general case, without an accompanying provenance mechanism shipped alongside it.

Disconfirmation. Falsified if systems handling permissioned, continuously updated, access-controlled corpora move predominantly to whole-corpus prompting and independent audits show claim-level attribution holding up at rates comparable to governed retrieval systems — that combination would mean the bookkeeping problem had been solved without a distinct retrieval stage, which is not what current evidence about positional use of long context suggests is happening.

What all four scenarios share, and what would break all four at once

Three things hold in every cell, which makes them the safest claims in this article.

First, corpus growth keeps a bounded fetch cheaper than a full paste, independent of model quality. The cost argument in the axes section does not depend on which cell the industry lands in; it depends only on corpora continuing to grow faster than windows shrink the marginal cost of using them in full, which every measured trend in context-length growth and corpus growth alike currently supports.

Second, a system’s location on Axis A does not determine its location on Axis B. Scenario 2 shows an explicit pipeline that stays unverifiable; Scenario 3 shows an implicit, dissolved pipeline that stays auditable via call-level logging. Anyone forecasting retrieval’s future from architecture alone — “agentic wins” or “long context wins” — is answering only Axis A and treating Axis B as though it followed automatically. It does not; it has to be engineered and paid for separately, and the vendor and standards evidence collected above shows that engineering is real but partial as of 2026.

Third, “retrieval versus long context” was never a single contest with one winner, because the two mechanisms are measured on different axes — accuracy under sufficient resourcing versus cost, not accuracy versus accuracy — and the strongest published comparison says so directly by proposing a router between them rather than a verdict ^[4]. Any account of 2035 that reports a clean winner between the two techniques is simplifying past what the source it would have to cite actually found.

The shared failure mode is symmetric with the shared strength. All four scenarios assume the two axes named here are the ones that matter. A development that changes what a “claim” or a “source” even means at the infrastructure level — for instance, if models begin reasoning primarily over structured, machine-verifiable knowledge representations rather than natural-language spans — would not sit cleanly in any of the four cells, because it would redefine what verifiability is being measured against rather than moving along either axis. None of the evidence gathered here suggests that shift is imminent, but a scenario framework is only as good as the axes it was built from, and this one names its own blind spot rather than pretending not to have one.

What to take away

Retrieval’s defining act was never storage; it was selection under a bound, followed by an argument built from what got selected. That act does not disappear in any of the four futures sketched here — even the Wide Window scenario still selects, implicitly, by whatever the context-assembly logic decides to include. What varies across the four cells is whether that selection stays visible as a named step, and whether a skeptical reader nine years from now could still ask “what was this claim actually based on” and get an answer that traces to a specific, checkable thing rather than to the system’s own unverified word.

The evidence collected here does not pick a cell, and it should not be read as leaning toward one. A vendor’s own documentation argues for pasting more into ever-larger windows; an independent benchmark shows that material in the middle of those windows is used worse than material at the edges; enterprise reporting shows hybrid retrieval adoption accelerating at the same moment a large share of enterprises report no production retrieval system at all; and a fast-moving agentic research literature argues the pipeline shape itself is already obsolete. Each of those is true, sourced, and pointing a different direction. The defensible 2035 forecast is the conditional one: wherever the economics in the axes section keep a bounded fetch cheaper than a full paste, something that functions like retrieval survives regardless of what it gets called, and whether anyone nine years from now can check its work depends on decisions about provenance that are being made, unevenly and mostly by accident, right now.