$$p(y \mid x) \approx \sum_{z \in \mathrm{top}\text{-}k\left(p_\eta(\cdot \mid x)\right)} p_\eta(z \mid x) \, p_\theta(y \mid x, z)$$

Equation 21 · AI Agents & Systems

From BM25 to Agentic Retrieval: A History of Retrieval-Augmented Generation

This equation gives an approximation: it relates the quantities while allowing an approximation.

By the time Lewis and colleagues published in 2020, learned dense retrieval and large pretrained generators both already existed as separate lines of work. The paper’s specific technical contribution was neither — it was making the whole pipeline trainable end to end through a single differentiable objective, treating the retrieved passage as a latent variable to be marginalized over rather than a fixed input handed to a frozen reader [ 8 ] . Writing x for the input, y for the output, z for a retrieved passage, and $\eta$ and $\theta$ for the retriever’s and generator’s parameters respectively, the RAG-Sequence variant approximates $p(y \mid x) \approx \sum_{z \in \mathrm{top}\text{-}k\left(p_\eta(\cdot \mid x)\right)} p_\eta(z \mid x) \, p_\theta(y \mid x, z)$. holding a single retrieved passage fixed…

Meanings in this article

• $x$: the writing.

Equation guide → · Article →