Symbol a

The passage around this formula

Formally, the supervisor’s policy chooses a subgoal for each worker, and each worker then runs its own instance of the general loop against a history that never touches its siblings’: $g^{(i)} \sim \pi_{\text{super}}(g \mid \text{task}), \qquad a_t^{(i)} \sim \pi_{\text{worker}}^{(i)}\bigl(a \mid o_{\le t}^{(i)},\ h_t^{(i)},\ g^{(i)}\bigr)$. Because $h^{(i)}$ and $h^{(j)}$ are disjoint by construction, worker i ’s error cannot enter worker j ’s context directly — it can only…

Read this part in the article →