Symbol s_n

The passage around this formula

A language model evaluation is, structurally, a function from a prompt to a string. Grading the string may be very hard, but the object being graded is small, static, and fully observed. An agent produces something categorically larger. The natural unit of observation is a trajectory: $\tau = \big(s_0,\ (a_1, o_1),\ \dots,\ (a_n, o_n),\ s_n\big)$. an initial environment state, an ordered sequence of actions with the observations they returned, and a terminal state. Everything that separates a competent run from a lucky one lives in the middle terms. A conventional success indicator reads only the last one.

Read this part in the article →