What does this equation mean?

Real corpora are full of near-duplicates: a press release syndicated across a dozen outlets, a policy document copied into three different wikis, boilerplate repeated at the top of every ticket in a queue. A ranking function that scores by similarity to the query has no reason to penalise duplication, so a query that matches one instance of a widely republished passage tends to match all of them, and a naive top- k selection can fill a majority of its slots with restatements of a single fact rather than covering the breadth of what is actually in the corpus. If d of the k retrieved slots are occupied by near-duplicates of an already-retrieved passage, the effective evidence budget is not k but $k_{\text{eff}} = k - d$. and d grows with corpus redundancy in a way that is invisible to any retrieval metric computed against a deduplicated relevance-judgement set, which is how most benchmarks are built. Schelpe’s recent empirical analysis, released as a preprint and not yet peer reviewed, measured exact-duplication rates across three regimes and found them to vary enormously by domain — under one percent in academic retrieval corpora, roughly a quarter of retrieved content in an enterprise document setting, and above four-fifths of retrieved content in a conversational-agent setting — while finding, across evaluation with several large language model providers, that removing exact duplicates before generation produced no measurable quality regression [ 12 ] . Read as a preliminary but suggestive result rather than a settled finding, it argues that deduplication is close to a free win precisely because the redundant slots were never doing any work.