$$Q(s,a) \leftarrow Q(s,a) + \alpha \left[ r + \gamma \max_{a'} Q(s',a') - Q(s,a) \right]$$

Equation 1 · Robotics & Embodied AI

Comparing the Main Approaches to Robotics and Embodied AI

This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.

A model-free update never references a transition model. A canonical form, temporal-difference learning, updates a value estimate purely from sampled transitions: $Q(s,a) \leftarrow Q(s,a) + \alpha \left[ r + \gamma \max_{a'} Q(s',a') - Q(s,a) \right]$. Nothing in this update requires knowing or estimating p(s' $\mid$ s, a) ; it only requires having experienced (s, a, r, s') . A model-based approach instead fits an explicit dynamics model $\hat{p}_\theta(s' \mid s, a)$ — a “world model” — and plans against it directly:

Equation guide → · Article →