Equation 10 · How a Model Actually Gets Small Enough to Run on a Phone
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
the square of T; tuned as a hyperparameter [ 1 ]. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
superscript
A raised number can be a power. When it is a label or bound, it selects a case or the upper limit of a sum; the formula’s structure distinguishes these uses.
See an illustrated explanation →How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
The factor is not decorative. Hinton and colleagues note that because the magnitude of the gradients produced by the soft-target term scales as 1/ , multiplying the term by keeps the relative contribution of the two loss terms roughly stable as T is tuned as a hyperparameter [ 1 ] . Without it, raising the temperature to expose more dark knowledge would simultaneously and silently shrink the gradient signal doing the exposing.
Sources cited in the surrounding passage
These citations give research context. Read each source to check which claims it supports.
Return to How a Model Actually Gets Small Enough to Run on a Phone