Symbol in context

$z_s$

This entry links every published equation using this exact notation. Its meaning may change between equations.

Meanings in context

Article-grounded definition

the teacher’s and student’s output logits.

z_s

This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.

\mathcal{L}_{\mathrm{distill}} = (1-\lambda)\,\mathcal{L}_{\mathrm{CE}}(y,\, \sigma(z_s)) + \lambda\, T^2 \, D_{\mathrm{KL}}\big(\sigma(z_t/T) \,\|\, \sigma(z_s/T)\big)

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

\mathcal{L}_{\text{distill}} = (1-\alpha)\,\mathrm{CE}\big(y,\ \sigma(z_s)\big) \;+\; \alpha\, T^2\,\mathrm{CE}\big(\sigma(z_t/T),\ \sigma(z_s/T)\big)

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

z_s

the teacher’s and student’s output logits.

z_s

This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.

\mathcal{L}_{\mathrm{KD}} = \alpha \, \mathcal{L}_{\mathrm{CE}}\left(y, \sigma(z_s)\right) + (1-\alpha)\, T^2 \, \mathrm{KL}\left(\sigma(z_t / T) \,\Vert\, \sigma(z_s / T)\right)

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.