$$b'(s') = \eta \, O(o \mid s', a) \sum_{s} T(s' \mid s, a)\, b(s)$$

Equation 9 · AI Agents & Systems

The Hardest Unsolved Problems in AI Agent Architecture

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

A useful decomposition for what an agent needs to track is the belief it holds over the true state of its task, updated as new observations arrive. In the classical formalism, if b(s) is the agent’s probability distribution over possible states before an observation, a is the action just taken, o is the observation that followed, T is the environment’s transition model and O its observation model, the updated belief is $b'(s') = \eta \, O(o \mid s', a) \sum_{s} T(s' \mid s, a)\, b(s)$. with $\eta$ a normalizing constant. Kaelbling, Littman and Cassandra’s original treatment of this update is also where the difficulty is made explicit: they show that the amount of memory an optimal policy needs is not bounded in advance by the size of the…

Meanings in this article

• $b$: the agent’s probability distribution over possible states before an observation.
• $s$: everything true about the task — every file changed, every external call made, every fact established —.
• $\eta$: the normalizing constant.
• $O$: the observation model.
• $o$: the observation that followed.
• $a$: the action just taken.
• $T$: the environment’s transition model.

Equation guide → · Article →