Symbol k

The passage around this formula

The latency consequence follows from where the reranker sits in the request path. A simplified but useful model of end-to-end latency is $L_{\mathrm{total}} \approx L_{\mathrm{retrieve}} + k \cdot \ell_{\mathrm{rerank}} + L_{\mathrm{generate}}$. where k is the number of candidates handed to the reranker and $\ell_{\mathrm{rerank}}$ is the marginal latency per candidate scored. Recall generally improves as k grows, because a larger candidate set is more likely to contain the passage that actually answers the query — but k multiplies the one latency…

Read this part in the article →