What does this equation mean?

By the time Lewis and colleagues published in 2020, learned dense retrieval and large pretrained generators both already existed as separate lines of work. The paper’s specific technical contribution was neither — it was making the whole pipeline trainable end to end through a single differentiable objective, treating the retrieved passage as a latent variable to be marginalized over rather than a fixed input handed to a frozen reader [ 8 ] . Writing x for the input, y for the output, z for a retrieved passage, and $\eta$ and $\theta$ for the retriever’s and generator’s parameters respectively, the RAG-Sequence variant approximates $p(y \mid x) \approx \sum_{z \in \mathrm{top}\text{-}k\left(p_\eta(\cdot \mid x)\right)} p_\eta(z \mid x) \, p_\theta(y \mid x, z)$. holding a single retrieved passage fixed across the whole generated sequence, while the RAG-Token variant allows the marginalization to be recomputed at every output token, letting different passages inform different parts of the answer. The retriever itself was a DPR-style dual encoder and the generator was BART; nothing about either component was new. What was new was the training signal running backward through the whole thing, so that the retriever’s parameters could be nudged by whether the passages it surfaced actually helped the generator, rather than being trained once, upstream, and frozen. Evaluated on knowledge-intensive tasks, the resulting models achieved state-of-the-art results and generated language the authors described as more specific, diverse, and factual than comparable parametric-only baselines [ 8 ] . The name retrieval-augmented generation describes this specific technical move — differentiable retrieval as part of a generation objective — not the broader retrieve-then-read pattern DrQA had already established three years earlier.