Equation 4 · Part 12 · How a Model Actually Gets Small Enough to Run on a Phone

Denominator: sum_j exp(z_j / T)

\sigma(z)_i = \frac{\exp(z_i / T)}{\sum_j \exp(z_j / T)}

\sum_j \exp(z_j / T)

What this part means

The complete quantity below the fraction bar; it must be nonzero for this division.

Its job in the formula

su $m_j$ exp( $z_j$ / T) occurs below the fraction bar. The quantity above the bar is divided by this expression; zero is excluded as a denominator.

Full expressionDenominator: sum_j exp(z_j / T)Article meaning

The passage around this formula

Distillation recovers it by training the small “student” network against the large “teacher” network’s full output distribution rather than against the label alone. Write the teacher’s and student’s pre-softmax outputs for a given input as $z_t$ and $z_s$ . A softmax with a temperature T is $\sigma(z)_i = \frac{\exp(z_i / T)}{\sum_j \exp(z_j / T)}$ . At T=1 this is the ordinary softmax used for prediction. Raising T flattens the distribution, pulling the small probabilities assigned to wrong classes up toward visibility, which is exactly the dark knowledge the method wants to expose. The training objective blends two terms: ordinary cross-entropy against the true label, and a match between the student’s and teacher’s…

Read this part in the article →

Learn the underlying idea

A fraction a/b means a divided by b. The top number is the numerator; the bottom number is the denominator, and it cannot be zero.

Open the illustrated fractions: division written vertically guide →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.