Equation 1 · One Model, Many Modalities: What Multimodal Systems Actually Share
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
This equation states an equality: the expressions on both sides have the same value under the article’s assumptions. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
Symbol N_tok
ok is part of the quantity the equation computes from the expression on the right.
Symbol P
P occurs below the fraction bar. The quantity above the bar is divided by this expression; zero is excluded as a denominator.
Symbol W
W occurs above the fraction bar. The numerator is divided by the entire denominator below it.
=
The expressions on both sides represent the same quantity under the stated assumptions.
See an illustrated explanation →subscript
The lower label selects a particular version, component, or indexed member of the quantity. For example, x₀ and xₜ can be values at different positions.
How to interpret it
With a fixed numerator, increasing a nonzero denominator reduces the fraction. Read it with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
For images the dominant answer is patchification, introduced at scale by the Vision Transformer: cut the image into a grid of non-overlapping square patches, flatten each, and project it linearly into the model’s embedding dimension, so that a pure transformer applied directly to sequences of image patches performs competitively with convolutional networks when pre-trained on enough data [ 2 ] . The token count follows immediately from the geometry, . for an image of height H and width W at patch size P . The quadratic relationship is the entire practical story of image tokenisation. Halving the patch size quadruples the sequence, and attention cost grows faster still.…
Read the full surrounding passage
For images the dominant answer is patchification, introduced at scale by the Vision Transformer: cut the image into a grid of non-overlapping square patches, flatten each, and project it linearly into the model’s embedding dimension, so that a pure transformer applied directly to sequences of image patches performs competitively with convolutional networks when pre-trained on enough data [ 2 ] . The token count follows immediately from the geometry, . for an image of height H and width W at patch size P . The quadratic relationship is the entire practical story of image tokenisation. Halving the patch size quadruples the sequence, and attention cost grows faster still. Every deployed system therefore resolves a three-way tension between input resolution, patch size, and context budget, and it resolves it by throwing away detail.
Sources cited in the surrounding passage
These citations give research context. Read each source to check which claims it supports.
Return to One Model, Many Modalities: What Multimodal Systems Actually Share