What does this equation mean?

That formula encodes a specific, testable assumption: that pairwise preference between any two systems can be explained by each system’s position on one shared scale, and that preferences are therefore transitive — if raters prefer A to B and B to C , they will prefer A to C . This is a real point of methodological disagreement rather than settled fact. Preferences over open-ended text plausibly depend on the task, the rater, and the specific pairing in ways a single scalar rating cannot capture, and a persistently observed intransitive preference cycle would be direct evidence against the model, not just noise around it. The Arena’s own published methodology treats this as an ongoing statistical concern to be monitored, not a solved problem.