← All parts of this equation

Equation 7 · Part 9 · How a Model Actually Gets Small Enough to Run on a Phone

Symbol T

LKD=α LCE(y,σ(zs))+(1−α) T2 KL(σ(zt/T) ∥ σ(zs/T))\mathcal{L}_{\mathrm{KD}} = \alpha \, \mathcal{L}_{\mathrm{CE}}\left(y, \sigma(z_s)\right) + (1-\alpha)\, T^2 \, \mathrm{KL}\left(\sigma(z_t / T) \,\Vert\, \sigma(z_s / T)\right)
TT

What this part means

tuned as a hyperparameter [ 1 ].

Its job in the formula

T is one of the signed contributions combined to compute the quantity on the left.

Where the article explains it

Hinton and colleagues note that because the magnitude of the gradients produced by the soft-target term scales as 1/T2T^2 , multiplying the term by T2T^2 keeps the relative contribution of the two loss terms roughly stable as T is tuned as a hyperparameter [ 1 ] .

The passage around this formula

At T=1 this is the ordinary softmax used for prediction. Raising T flattens the distribution, pulling the small probabilities assigned to wrong classes up toward visibility, which is exactly the dark knowledge the method wants to expose. The training objective blends two terms: ordinary cross-entropy against the true label, and a match between the student’s and…

Read this part in the article →

Learn the underlying idea

A variable is a named place for a value. Its letter is a local label: x can mean position in one formula and a data point in another.

Open the illustrated variables: a letter stands for a value guide →

See this notation across published equations →

Sources cited in the surrounding passage

These citations provide research context; check each source for the exact claim it supports.