Symbol f

The passage around this formula

…JumpReLU, a thresholded activation function with a learned per-feature cutoff $\theta$ trained through a straight-through gradient estimator, which lets the objective penalise the true count of active features, $\lVert$ f(x) $\rVert_0$ , directly rather than through the $\ell_1$ proxy, and reported state-of-the-art reconstruction fidelity at matched sparsity against both the earlier $\ell_1$ formulation and a competing gated variant [ 7 ] . That the field kept revising the sparsity penalty is itself evidence of how much…

Read this part in the article →