Symbol w_ij
j is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Read this term in its guide →Published equation contexts
Why this formula appears here
Distillation decides how the student’s weights are trained. Pruning decides which weights exist afterward. The oldest and simplest criterion is magnitude: rank every weight by || and remove the smallest fraction, on the reasoning that a weight close to zero contributes almost nothing to the layer’s output no matter what it is multiplied against. Applied once, magnitude pruning at any aggressive ratio breaks the network; applied iteratively, alternating a pruning step with a retraining step that lets the surviving weights absorb what the removed ones were doing, it can remove a large majority of parameters with only a small accuracy cost.
Read the representative guide
How to interpret it
Read this expression with the definitions, units, and assumptions supplied by the article.
Research cited beside this formula
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 14 · Edge AI & Electronics
How a Model Actually Gets Small Enough to Run on a Phone
This mathematical expression combines the displayed quantities; its precise role follows from the surrounding article text.
Distillation decides how the student’s weights are trained. Pruning decides which weights exist afterward. The oldest and simplest criterion is magnitude: rank every weight by || and remove the smallest fraction, on the reasoning that a weight close to zero contributes almost nothing to the layer’s output no matter what it is multiplied against. Applied once, magnitude pruning at any aggressive ratio breaks the network; applied iteratively, alternating a pruning step with a retraining step that lets the surviving weights absorb what the removed ones were doing, it can remove a large majority of parameters with only a small accuracy cost.
Equation guide → · Article →