$$P(\text{correct}) \;=\; \underbrace{P(R)\,P(\text{correct} \mid R)}_{\text{grounded in retrieved evidence}} \;+\; \underbrace{P(\lnot R)\,P(\text{correct} \mid \lnot R)}_{\text{correct without sufficient retrieved evidence}}$$

Equation 19 · AI Agents & Systems

Measuring What a RAG System Retrieves, Not Just What It Answers

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

The measurement structure explains the divergence directly. Let R be the event that the retriever’s top- k results contain sufficient evidence to answer the query, and let “correct” mean the generated answer is judged faithful and relevant. Total correctness decomposes as $P(\text{correct}) \;=\; \underbrace{P(R)\,P(\text{correct} \mid R)}_{\text{grounded in retrieved evidence}} \;+\; \underbrace{P(\lnot R)\,P(\text{correct} \mid \lnot R)}_{\text{correct without sufficient retrieved evidence}}$. An end-to-end accuracy score measures the left-hand side only. The two terms on the right are exactly what generation metrics and retrieval metrics separately illuminate, and the second term is the one an aggregate score cannot see at all. P($\text{correct}$ $\mid$ $\lnot$ R) is the probability the model answers correctly despite the retriever having failed — because the underlying language model already…

Meanings in this article

• $R$: the event that the retriever’s top- k results contain sufficient evidence to answer the query, and let “correct” mean the generated answer is judged faithful and relevant.

Equation guide → · Article →