≤

The passage around this formula

Strip away the branding and every one of these systems is choosing an action from a history of observations rather than from the true state of the world, because the true state is never directly available to it. That is the founding move of decision theory under partial observability: an agent’s policy must be a function of a belief built from what it has seen, not of the world as it actually is, and the two can silently diverge [ 11 ] . Write the general shape once, for a single decision-maker at a single step: $a_t \sim \pi_\theta(a \mid o_{\le t},\ h_t,\ g)$. where $o_{\le t}$ is the observation history, $h_t$ is whatever state the system has chosen to retain, and g is the task goal. Nothing here is specific to language…

Read this part in the article →