Equation 10 · Part 6 · A History of Small and On-Device AI
=
=
What this part means
The expressions on both sides represent the same quantity under the stated assumptions.
Its job in the formula
The equals sign connects the complete expression on the left with the complete expression on the right. Both sides must have compatible units.
Full expression→=→Article meaning
The passage around this formula
Hinton, Vinyals and Dean’s 2015 paper, “Distilling the Knowledge in a Neural Network,” is the one the field actually cites, and it earned that position by generalizing the 2006 idea and giving it a mechanism simple enough to fit into any neural network’s training pipeline. Instead of training a small model against a large model’s hard output labels, distillation trains it against the large “teacher” model’s full, softened probability distribution over classes — the softmax output computed at a raised temperature T : . where are the teacher’s pre-softmax logits. “Using a higher value for T produces a softer probability distribution over classes” [ 4 ] , and it is…
Learn the underlying idea
An equals sign says that the expression on its left and the expression on its right have the same value under the stated definitions and assumptions.
Open the illustrated equality: what the equals sign claims guide →
Sources cited in the surrounding passage
These citations provide research context; check each source for the exact claim it supports.