fraction

The passage around this formula

tokens. Video adds a third factor. If a model attends over T sampled frames, each cut into the same spatial grid, and groups $P_t$ consecutive frames per temporal patch, the token count becomes $N_{\mathrm{tok}} = \frac{H}{P} \cdot \frac{W}{P} \cdot \frac{T}{P_t}$. Every added frame multiplies the spatial cost rather than adding to it. A minute of video at even a modest frame rate is not “a bigger picture” in the way a higher-resolution photograph is; it is a volume, and attention cost over a transformer sequence grows faster than the sequence itself. No system deployed today attends densely over every raw frame of a long clip, because nothing could afford to.

Read this part in the article →