Equation 3 · Part 10 · How to Evaluate Codex Beyond SWE-bench and Vendor Scores
Starting index or lower bound: i=1
What this part means
This label says where the repeated addition, multiplication, or accumulation starts. Read its value or condition together with the article’s description of the index.
Its job in the formula
i=1 appears in the bound of this sum. The bound states where the repeated operation starts, ends, or which values it includes.
Full expression→Starting index or lower bound: i=1→Article meaning
The passage around this formula
Let \{0,1\} indicate whether evaluation task i is resolved. The familiar score . is an estimate of success probability only for a distribution represented by those n instances and the evaluated system. The system is not just a model. It includes context retrieval, tools, prompts, retries, execution limits, and candidate selection. The evaluation distribution is not “software engineering.” It is a constructed sample with inclusion criteria, repository mix, languages, issue styles, and executable tests.
Learn the underlying idea
Σ adds a collection of terms. Π multiplies them. The lower and upper labels tell you which terms belong to the collection.
Open the illustrated sums and products: repeat an operation over an index guide →
The article lists its research sources here.