Naive, Graph, and Agentic: A Systems Comparison of RAG Architectures

Five things sharing one name

“Retrieval-augmented generation” names a design pattern, not a design. Say a system “uses RAG” and you have specified almost nothing about its engineering: how many times it calls a retriever per request, whether it calls a retriever at all before deciding to, what data structure it searches, or whether the number of retrieval calls is fixed at build time or decided at run time by the model itself. Five families answer these questions differently enough that they belong in separate rows of an architecture diagram, not under one label.

The five, in the order this article takes them: naive single-pass retrieve-then-generate, the architecture Lewis and colleagues formalised as a combination of a parametric generator and a non-parametric retriever ^[1]; hybrid retrieval that runs sparse and dense search in parallel and fuses or reranks the result; graph-augmented retrieval that builds an indexed knowledge structure over the corpus before any query arrives, of which Microsoft’s GraphRAG is the most visible instance ^[4]; iterative or multi-hop retrieval that interleaves fetching with reasoning across several rounds; and agentic retrieval in which a model, rather than a fixed pipeline, decides at run time whether to retrieve, what to retrieve, and when it has retrieved enough.

Two framings recur across the literature on how these fit together. Gao and colleagues’ widely cited survey groups the field into Naive, Advanced, and Modular RAG, treating everything past the naive baseline as elaborations on a shared pipeline of indexing, retrieval, and generation ^[11]. Singh and colleagues instead draw the line at agency itself, proposing a taxonomy of agentic RAG organised around agent cardinality, control structure, autonomy, and how knowledge is represented ^[13]. Neither framing is wrong; they are cutting the same space along different axes, and this article borrows from both without adopting either exclusively.

What follows treats each family as a genuine engineering artefact: what it costs to build, what it costs per query, where it fails, and what kind of question it was actually built to answer. The comparison that closes the article is deliberately not a leaderboard. Every family below reports its evidence on a different benchmark, assembled by a different team, under a different harness — HotpotQA-style multi-hop QA is not the same measurement as query-focused summarisation over a million-token corpus, and neither is comparable to an agent-loop benchmark scored on tool-call efficiency. Holding five such numbers next to each other and calling the largest one a winner would misrepresent every paper cited here. The axes that follow are engineering axes, not accuracy axes, for exactly that reason.

The naive baseline: retrieve once, generate once

Start with the architecture that gives the pattern its name. A query is encoded, a retriever returns its top-$k$ passages against a fixed index, and a generator conditions on those passages to produce one answer. No step revisits an earlier one. Lewis and colleagues’ original formulation makes the mechanism explicit: writing $x$ for the query, $y$ for the output, and $z$ for a retrieved passage drawn from a top-$k$ set $\mathcal{Z}_k(x)$, the model marginalises over that set,

with retriever parameters $\eta$ and generator parameters $\theta$ ^[1]. One retrieval call, one generation pass, and the entire system’s dependence on the corpus runs through that single set $\mathcal{Z}_k(x)$.

The engineering profile that follows from this shape is easy to state and easy to underestimate. Latency is the sum of one retrieval lookup and one generation call, both boundable and both measurable in isolation, which makes the naive architecture the cheapest and most predictable of the five to run in production — its tail latency is close to its median latency, because there is no loop that a hard query can make longer. Its infrastructure footprint is a single index and a single retriever, with no reranking stage, no graph store, and no orchestration layer to operate or monitor. Its failure surface is correspondingly narrow but unforgiving: if the one retrieval call misses the passage the question needs, there is no second attempt built into the architecture, and the generator will produce a confident answer from whatever it was given regardless of whether that was sufficient. Questions whose answer requires combining facts scattered across more than one passage are structurally outside what a single top-$k$ fetch can supply, independent of how good the retriever is.

That last limitation is not a bug to be tuned away; it is the shape of the architecture. A single retrieval call returns a fixed set before the generator has produced a single token, so nothing the generator subsequently reasons its way toward can influence what was fetched. For a workload of independent, single-fact lookups against a stable corpus — a support-documentation assistant, a product-catalogue question-answerer — this is close to the correct amount of machinery: fast, cheap, and auditable, because the entire evidential basis for an answer is the one retrieved set, visible in full.

Hybrid retrieval and reranking: buying accuracy with an extra hop

The first elaboration on the naive baseline keeps the single-pass shape — one retrieval stage feeding one generation call — but replaces a single retriever with two running in parallel, plus a step to reconcile their outputs.

The two retrievers fail in different directions, which is the entire justification for running both. Lexical retrieval, the BM25 family, scores documents by term overlap and fails on vocabulary mismatch: a query about “cannot log in” scores nothing against a document that says “authentication failure”. Dense retrieval, the family Karpukhin and colleagues established with a dual-encoder trained on question-passage pairs, embeds query and passage into a shared vector space and fails on precision, losing rare identifiers — part numbers, version strings, error codes — that carry little weight in a learned embedding but enormous discriminative value lexically; their dense retriever nonetheless reported nine to nineteen percentage points of absolute improvement in top-20 retrieval accuracy over a BM25 baseline on open-domain question answering, evidence that neither family dominates the other in general ^[2]. Fusing the two recovers most of each one’s strength. Reciprocal rank fusion, which Cormack and colleagues introduced and showed outperforming both individual rankers and a Condorcet-style combination, ignores raw scores entirely and combines rank positions across retrievers $R$ with a smoothing constant $k$,

sidestepping the problem that a BM25 score and a cosine similarity are not directly comparable quantities ^[3]. A reranking stage — typically a cross-encoder scoring each retrieved candidate jointly with the query — can then sit downstream of the fused list and reorder it more precisely than either upstream retriever could alone.

The systems cost of this family is a second retrieval index to build and keep synchronised with the first, plus a reranking pass whose latency is paid on every single request regardless of whether that request needed the extra precision. Wang and colleagues’ large-scale empirical study of RAG module combinations found reranking to be one of the more consistently valuable additions in their evaluation, while stressing that most stages in a RAG workflow trade some combination of latency, cost, and quality against each other rather than offering a free improvement, and that the workflow choices interact rather than being separately optimal ^[12]. The honest characterisation of this family is that it buys retrieval precision at the cost of a fixed, per-request latency tax and a second piece of infrastructure to operate — a trade that is worth making whenever retrieval accuracy, not generation, is the binding constraint on answer quality, and a needless one when it is not.

Graph-augmented retrieval: paying at index time to answer differently shaped questions

The naive and hybrid families both treat the corpus as an unstructured pool of chunks. Graph-augmented retrieval instead builds a structure over the corpus before any query exists: entities and relations are extracted, organised into a graph, and — in the specific approach Edge and colleagues describe as GraphRAG — clustered into a hierarchy of communities, each summarised by an LLM call made once at index time ^[4]. A query at run time can then be answered in one of several ways depending on its shape. Microsoft’s documentation for the project describes local search, which walks outward from specific named entities and their direct neighbours; global search, which reasons over the community summaries to answer questions about the corpus as a whole; and DRIFT search, which combines the two ^[5].

This is the one family among the five whose defining engineering trade-off is temporal rather than architectural: cost is shifted from query time to index time. Writing $C_{\text{index}}$ for the one-time cost of building the graph and its community summaries, and $c_{\text{marginal}}$ for the cost of answering a single query against the finished structure, the amortised cost of a query workload of size $Q$ against one graph is

When $Q$ is small — a corpus queried rarely, or one still being explored — $C_{\text{index}}$ dominates and the naive or hybrid architectures are cheaper by a wide margin, because they pay nothing until a query arrives. When $Q$ is large against a stable corpus, the fixed term is driven down toward zero and the comparison turns on $c_{\text{marginal}}$ against the per-query cost of the alternatives. The direction of that comparison depends on the question being asked, not only on volume: Edge and colleagues report that GraphRAG produces substantially more comprehensive and diverse answers than a conventional RAG baseline specifically on query-focused summarisation over large corpora, the kind of question — “what are the main themes across this entire collection” — that a top-$k$ chunk retriever structurally cannot answer well no matter how many times it is queried, because no fixed small set of chunks represents a corpus-wide theme ^[4].

$C_{\text{index}}$ is also the family’s most visible weakness, and the project’s own subsequent work treats it as one worth solving rather than downplaying, which is worth reading as a vendor’s own admission of the problem’s severity. Microsoft’s follow-up post introducing LazyGraphRAG reports — as a vendor claim about the vendor’s own systems, not an independently audited benchmark — that the original approach’s indexing cost was steep enough to motivate a redesign, and states that LazyGraphRAG’s indexing cost is on par with plain vector-based RAG and roughly one part in a thousand of full GraphRAG’s, while its global-search query cost is reported as roughly seven hundred times lower than the original for comparable answer quality, or about four percent of the original’s query cost at a stated evaluation budget ^[6]. Read alongside the amortised-cost model above, this is a description of the same $C_{\text{index}}$ term being pushed down by engineering rather than by amortisation, which is a different lever entirely and one the other four families do not have available in the same form, since none of them pays a comparable fixed cost to begin with.

The corpus-maintenance cost this family carries is structural rather than a mere implementation detail worth flagging once: a graph built over a corpus goes stale exactly as the corpus changes, and unlike a chunk index — which can be updated by adding or removing individual vectors — a knowledge graph’s community structure is a global property of the whole corpus, so a substantial edit can in principle require re-deriving summaries that depended on it. This is the family’s most workload-defining property: it fits a corpus that is large, relatively stable, and queried repeatedly with questions that genuinely require synthesis across many documents rather than retrieval of one or two passages.

Iterative and multi-hop retrieval: when one fetch cannot answer the question

A different failure of the naive architecture motivates a different fix. Some questions cannot be answered from any single retrieved set because answering them requires knowing the answer to a sub-question first. “What is the capital of the country where the director of Film X was born” cannot be resolved by one query against a passage index unless that exact composite fact happens to be written down somewhere; it requires finding the director, then their birth country, then that country’s capital, each step depending on the result of the one before it.

Trivedi and colleagues’ IRCoT addresses this directly by interleaving retrieval with chain-of-thought reasoning rather than performing either in one shot: at each step, a reasoning sentence is generated conditioned on the question and everything retrieved so far, that sentence is used as the next retrieval query, and the newly retrieved passages are added to what the next reasoning step can draw on. Their framing of the dependency is precise: what to retrieve depends on what has already been derived, which in turn depends on what was previously retrieved, so the two cannot be separated into one retrieval phase followed by one reasoning phase ^[7]. Evaluated across HotpotQA, 2WikiMultihopQA, MuSiQue, and IIRC — all purpose-built multi-hop benchmarks — they report gains of up to twenty-one points in retrieval performance and up to fifteen points in downstream QA accuracy over one-shot retrieval baselines, alongside reduced hallucination ^[7]. Jiang and colleagues’ FLARE takes a related but distinct approach, retrieving reactively rather than at fixed steps: it monitors the confidence of the tokens being generated and triggers a fresh retrieval specifically when the model’s own generation becomes uncertain, then regenerates the low-confidence span with the newly retrieved evidence in context ^[8].

Both approaches share the same engineering consequence: the number of retrieval-generation rounds, call it $k$, is no longer fixed at one. Latency and cost scale with $k$, and unlike the naive or hybrid families, $k$ is now a property of the question rather than of the architecture — a single-hop question still costs roughly one round, but a four-hop question costs roughly four, and the system pays that multiplier whether or not the caller anticipated it. IRCoT bounds this in practice by capping the number of reasoning steps and by using retrieval and reasoning specifically tuned for the multi-hop benchmarks it was evaluated on; FLARE bounds it by triggering additional retrieval only on measured low confidence rather than at every step, which keeps the multiplier close to one on easy inputs and lets it grow on hard ones. Neither of these figures — the reported point gains on multi-hop QA benchmarks — is comparable to GraphRAG’s comprehensiveness gains on corpus summarisation or to Self-RAG’s citation-accuracy figures discussed next; they were measured on different tasks built to expose different failures, and citing one as evidence against another would be a category error, not a finding.

Agentic RAG: retrieval as a decision, not a step

The final family removes the fixed pipeline shape entirely. Rather than a predetermined sequence of retrieval and generation steps — one round for naive RAG, a small bounded number for iterative RAG — an agentic system has a model decide, at each point in a longer interaction, whether to retrieve at all, what query to issue if it does, and when it has gathered enough to answer.

Two lines of prior work underpin this. Yao and colleagues’ ReAct is the foundational pattern, not a RAG-specific method: it has a language model generate interleaved reasoning traces and actions, where an action can be any tool call — including, in their question-answering experiments, a Wikipedia search — and the reasoning trace helps the model track what it still needs and decide what to do next, reporting reduced hallucination on question-answering and fact-verification tasks relative to reasoning-only or acting-only baselines ^[10]. Asai and colleagues’ Self-RAG specialises this idea for retrieval specifically: the model is trained to emit reflection tokens that mark whether retrieval is needed at a given point, whether a retrieved passage is relevant, and whether the resulting generation is actually supported by it, making retrieval “adaptive” in the sense that the model — not a fixed schedule — decides when to invoke it, and the same tokens let the model critique its own output for factual support after the fact; their 7B and 13B models are reported to outperform substantially larger instruction-tuned baselines on open-domain QA, reasoning, and fact-verification benchmarks, with particular gains in citation accuracy on long-form generation ^[9].

Singh and colleagues’ survey names the resulting engineering problem precisely: an agentic system’s retrieval behaviour is now controlled by a learned or prompted policy rather than by a fixed pipeline parameter, and their taxonomy organises the resulting architectures by how many agents are involved, how they are coordinated, how much autonomy each has, and how the underlying knowledge is represented and accessed ^[13]. The engineering consequence of policy-controlled retrieval is the one this family cannot avoid: the round count $k$ from the iterative family’s cost model is, for an agentic system, a random variable whose distribution depends on the policy and the specific input, not a number fixed in advance by the architecture or bounded by a benchmark-tuned heuristic. Absent an operator-imposed budget — a maximum number of tool calls, a wall-clock timeout, a token cap — nothing in the architecture itself guarantees $k$ terminates quickly, or terminates at all on a pathological input. This is the direct cost of the family’s chief advantage, which is real: retrieval effort is matched to the difficulty of the specific request rather than spent uniformly, so an easy question can be answered in one round at close to naive-RAG cost while a hard one draws on as many rounds as its policy judges necessary — but “as many as necessary” is a claim about the policy’s competence, not a guarantee, and the failure mode when the policy misjudges is an unpredictable latency and cost tail rather than the cleanly bounded worst case the other four families offer.

A systems comparison, not a leaderboard

Collect the five families’ engineering shape in one place, on the axes that generalise across them rather than the accuracy numbers that do not.

A single latency model makes the difference between the first three rows and the last two concrete. Writing $L_{\text{enc}}$ for query encoding, and $L_{\text{retrieve}}^{(i)}$ and $L_{\text{LLM}}^{(i)}$ for the retrieval and generation cost of round $i$,

For naive and hybrid RAG, $k = 1$ by construction — the sum has exactly one term, and total latency is boundable in advance for any query. For iterative RAG, $k$ is a small integer set by a heuristic or a step cap chosen by the system builder, so the worst case is known even though the typical case varies with question difficulty. For agentic RAG, $k$ is a random variable generated by the policy itself at run time, and its distribution is exactly what the operator does not control without adding an external cap. The same formula describes all five families; what changes is only where $k$ comes from, and that difference is the whole of the engineering distinction between “a pipeline with a loop in it” and “a policy that decides how much to loop.”

None of this licenses ranking the families by the accuracy numbers reported in the papers above. IRCoT’s fifteen-to-twenty-one-point gains were measured on HotpotQA-family multi-hop benchmarks against one-shot retrieval baselines ^[7]. GraphRAG’s comprehensiveness and diversity gains were measured on query-focused summarisation over roughly million-token corpora against a conventional RAG baseline ^[4]. Self-RAG’s citation-accuracy gains were measured on long-form generation against substantially larger non-adaptive baselines ^[9]. These are three different tasks, three different baselines, and three different evaluation harnesses; nothing in this article’s account should be read as implying that a 21-point retrieval gain on multi-hop QA is “bigger” or “better” than a comprehensiveness win on corpus summarisation. They are not on the same scale, and the correct response to seeing them printed near each other is to ask what each benchmark can and cannot tell you about your own workload, not to average them.

Choosing an architecture for a workload

The comparison above resolves into a small number of practical questions worth asking before committing to any of the five.

Does the corpus fit inside a stable, well-scoped index, and are queries independent lookups rather than compositional ones? If so, the naive architecture is not a placeholder for something more sophisticated — it is very often the correct answer, and every family past it is buying a specific capability at a specific, measurable cost that a workload without the corresponding failure mode does not need to pay.

Is retrieval precision, specifically, the observed bottleneck — rare identifiers dropped by a dense retriever, or paraphrased queries missed by a lexical one? Hybrid retrieval with a tuned fusion function and, if the latency budget allows a reranking pass, addresses that bottleneck directly, at the fixed per-request cost described above.

Does the workload ask corpus-wide questions — themes, summaries, connections between entities that are never stated together in any single passage — against a corpus large and stable enough to make an amortised index-time cost worthwhile? That is the shape graph-augmented retrieval was built for, and the amortisation model above gives a concrete way to reason about whether a given query volume against a given corpus clears the fixed cost of building the structure.

Does answering a typical question require deriving an intermediate fact before knowing what to retrieve next? That is the specific failure that motivates iterative and multi-hop retrieval, and it is worth confirming the failure is actually present — most production question sets are not multi-hop, and adding a retrieval loop to a workload that does not need one only adds the latency and cost of the extra rounds without a corresponding accuracy gain.

Finally, does the interaction span multiple turns with retrieval needs that cannot be anticipated at design time — an assistant working through an open-ended task where what to look up next depends on what was just found? That is the case for agentic retrieval, and it is also the case where an operator-imposed budget on rounds, tool calls, or wall-clock time is not an optional safeguard but the mechanism that keeps the family’s chief advantage from becoming its chief liability.

Predictions, with the observations that would falsify them

These are forecasts, clearly separated from the sourced comparison above. Horizon: 11 August 2028.

One. Production agentic RAG deployments will converge on explicit, operator-set budgets — maximum tool calls, wall-clock timeouts, or token caps — as a default rather than an optional safeguard, because the unbounded round count described above will keep producing cost incidents severe enough to force the issue. Disconfirmed if mainstream agent frameworks in 2028 still ship agentic retrieval loops with no default bound on round count.

Two. Graph-augmented retrieval’s indexing cost will continue to fall through engineering rather than through amortisation, following the trajectory the LazyGraphRAG work already shows, and the fixed-versus-marginal cost trade-off will shift measurably toward index-time cost mattering less. Disconfirmed if graph-index construction cost, measured per token of source corpus, is not materially lower in 2028 than the costs reported for the original 2024 GraphRAG approach.

Three. No single accuracy leaderboard spanning all five families will emerge as a trusted default for architecture selection, because the families answer structurally different question types and a shared benchmark would have to flatten that difference to produce one ranking. Disconfirmed if a benchmark combining naive, hybrid, graph, iterative, and agentic RAG under one harness and one headline score becomes a standard citation for choosing among them.

Four. Hybrid sparse-dense retrieval with a tuned fusion function will remain present, in some form, inside graph-augmented and agentic systems as their base retrieval layer, rather than being displaced by pure dense or pure graph lookup, because the vocabulary-mismatch and precision failures it addresses do not depend on which higher-level architecture sits above it. Disconfirmed if leading graph-augmented or agentic RAG frameworks in 2028 predominantly use single-method retrieval as their base layer.

None of these requires a capability discontinuity. They follow from the structure already visible: five architectures whose retrieval round count is fixed, prebuilt, heuristic-bounded, or policy-determined, each trade governed by where the cost of that round count is paid and by what kind of question it was built to answer.

What to take away

“Which RAG architecture is best” is not a question with an answer, for the same reason “which vehicle is best” is not — a bicycle, a delivery van, and a freight train are not competing on one axis, and asking which wins misdescribes the choice being made. Naive RAG is one retrieval call and one generation call, cheap and bounded, correct for independent lookups against a stable corpus. Hybrid retrieval adds a second index and a fusion or reranking stage to fix a specific, diagnosable precision problem, at a fixed per-request cost. Graph-augmented retrieval moves a large cost from query time to index time to answer corpus-wide questions no chunk retriever can reach, and that fixed cost is currently falling through active engineering effort rather than through query volume alone. Iterative retrieval turns the fixed single round into a small, bounded number of rounds to answer compositional questions a single fetch cannot. Agentic retrieval removes the bound entirely and hands round-count control to a policy, buying adaptivity at the cost of a latency and expense tail that only an operator-imposed budget keeps in check.

Treat each family as an answer to a specific engineering question, not as a rung on a single ladder of sophistication. The five families cited here were evaluated on five different tasks by five different teams, and no honest reading of their published numbers ranks them against each other. The only ranking that means anything is the one produced by measuring your own corpus, your own query distribution, and your own latency and cost budget against each architecture’s actual shape — and that measurement, unlike any of the benchmark scores above, is one nobody has run for you yet.