subscript

The passage around this formula

Formally, the supervisor’s policy chooses a subgoal for each worker, and each worker then runs its own instance of the general loop against a history that never touches its siblings’: $g^{(i)} \sim \pi_{\text{super}}(g \mid \text{task}), \qquad a_t^{(i)} \sim \pi_{\text{worker}}^{(i)}\bigl(a \mid o_{\le t}^{(i)},\ h_t^{(i)},\ g^{(i)}\bigr)$. Because $h^{(i)}$ and $h^{(j)}$ are disjoint by construction, worker i ’s error cannot enter worker j ’s context directly — it can only reach the supervisor, and only in whatever compressed form the worker chooses to report. That disjointness is the formal reason this pattern offers stronger containment than the single loop for genuinely independent subtasks, and it is also exactly why the pattern is documented to struggle when subtasks are not independent: the architecture has no channel for…

Read this part in the article →