What does this equation mean?

A useful decomposition for what an agent needs to track is the belief it holds over the true state of its task, updated as new observations arrive. In the classical formalism, if b(s) is the agent’s probability distribution over possible states before an observation, a is the action just taken, o is the observation that followed, T is the environment’s transition model and O its observation model, the updated belief is $b'(s') = \eta \, O(o \mid s', a) \sum_{s} T(s' \mid s, a)\, b(s)$. with $\eta$ a normalizing constant. Kaelbling, Littman and Cassandra’s original treatment of this update is also where the difficulty is made explicit: they show that the amount of memory an optimal policy needs is not bounded in advance by the size of the problem, and that reducing the reliability of a single observation channel in their own worked example forces a much larger plan graph, and a correspondingly larger memory requirement, just to stay confident [ 4 ] . Applied to a long-running LLM agent, s is everything true about the task — every file changed, every external call made, every fact established — and b is whatever the agent’s context window currently represents. The update above is exact only if the full state and its full history are kept; an agent cannot keep them, because the context window is finite and every token inside it costs money and attention.