← All parts of this equation

Equation 12 · Part 6 · The Hardest Unsolved Problems in Small and On-Device AI

Symbol F_i

L(θ)=Lnew(θ)+λ∑iFi(θi−θi∗)2,\mathcal{L}(\theta) = \mathcal{L}_{\mathrm{new}}(\theta) + \lambda \sum_{i} F_i\left(\theta_i - \theta_i^{*}\right)^2,
FiF_i

What this part means

the computing and storing.

Its job in the formula

FiF_i is one of the signed contributions combined to compute the quantity on the left.

Where the article explains it

It also exposes a cost specific to on-device deployment: computing and storing FiF_i for every parameter, and doing so repeatedly as the device keeps learning, is itself memory and compute that a phone-class budget has to find room for, on top of whatever the update itself costs.

The passage around this formula

…is regularization: penalize the optimizer for moving parameters that mattered to earlier tasks. The best-known form estimates a per-parameter importance weight — commonly the diagonal of the Fisher information, FiF_i — from the old task, and adds it to the new loss: L(θ)=Lnew(θ)+λ∑iFi(θi−θi∗)2\mathcal{L}(\theta) = \mathcal{L}_{\mathrm{new}}(\theta) + \lambda \sum_{i} F_i\left(\theta_i - \theta_i^{*}\right)^2. where θi∗\theta_i^{*} is the old optimum for parameter i and λ\lambda sets how strongly the old task is protected. This equation exposes the actual trade rather than resolving it: raising λ\lambda protects old knowledge at the…

Read this part in the article →

Learn the underlying idea

A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.

Open the illustrated subscripts: which member of a family? guide →

See this notation across published equations →

Sources cited in the article section

These citations provide research context; check each source for the exact claim it supports.