=

The passage around this formula

A single latency model makes the difference between the first three rows and the last two concrete. Writing $L_{\text{enc}}$ for query encoding, and $L_{\text{retrieve}}^{(i)}$ and $L_{\text{LLM}}^{(i)}$ for the retrieval and generation cost of round i , $L_{\text{total}} = L_{\text{enc}} + \sum_{i=1}^{k} \left( L_{\text{retrieve}}^{(i)} + L_{\text{LLM}}^{(i)} \right)$. For naive and hybrid RAG, k = 1 by construction — the sum has exactly one term, and total latency is boundable in advance for any query. For iterative RAG, k is a small integer set by a heuristic or a step cap chosen by the system builder, so the worst case is known even though the typical case varies with question difficulty. For agentic RAG, k is a random variable generated by the policy itself at run time, and its…

Read this part in the article →