Equation 5 · Part 1 · How a Model Actually Gets Small Enough to Run on a Phone
Symbol T
What this part means
T is part of the quantity the equation computes from the expression on the right.
Its job in the formula
T is part of the quantity the equation computes from the expression on the right.
Full expression→Symbol T→Article meaning
The passage around this formula
At T=1 this is the ordinary softmax used for prediction. Raising T flattens the distribution, pulling the small probabilities assigned to wrong classes up toward visibility, which is exactly the dark knowledge the method wants to expose. The training objective blends two terms: ordinary cross-entropy against the true label, and a match between the student’s and teacher’s temperature-softened distributions, measured by KL divergence:
Learn the underlying idea
A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.
Open the illustrated variables: a letter stands for a value guide →
See this notation across published equations →
Sources cited in the article section
- [1] Distilling the Knowledge in a Neural Network ↗
- [14] A Survey on Knowledge Distillation of Large Language Models ↗
These citations provide research context; check each source for the exact claim it supports.