← Mathematical compendium

Published equation contexts

L(θ)=Lnew(θ)+λ∑iFi(θi−θi∗)2\mathcal{L}(\theta) = \mathcal{L}_{\mathrm{new}}(\theta) + \lambda \sum_{i} F_i\left(\theta_i - \theta_i^{*}\right)^2

Why this formula appears here

One standard mitigation is regularization: penalize the optimizer for moving parameters that mattered to earlier tasks. The best-known form estimates a per-parameter importance weight — commonly the diagonal of the Fisher information, FiF_i — from the old task, and adds it to the new loss: L(θ)=Lnew(θ)+λ∑iFi(θi−θi∗)2\mathcal{L}(\theta) = \mathcal{L}_{\mathrm{new}}(\theta) + \lambda \sum_{i} F_i\left(\theta_i - \theta_i^{*}\right)^2. where θi∗\theta_i^{*} is the old optimum for parameter i and λ\lambda sets how strongly the old task is protected. This equation exposes the actual trade rather than resolving it: raising λ\lambda protects old knowledge at the direct expense of how much the new update is allowed to change the model, and there is no value of λ\lambda that removes the trade — only one that relocates it. It also…

Read the full article-specific guide →

Read the representative guide

θ\theta

Symbol θ

θ is an argument of the function-like quantity on the left; its role is set by that function’s stated inputs.

Read this term in its guide →
Lnew\mathcal{L}_{\mathrm{new}}

Symbol L_new

LnL_new is one of the signed contributions combined to compute the quantity on the left.

Read this term in its guide →
ii

Symbol i

i appears in the bound of this sum. The bound states where the repeated operation starts, ends, or which values it includes.

Read this term in its guide →
θi\theta_i

Symbol theta_i

thetaia_i is one of the signed contributions combined to compute the quantity on the left.

Read this term in its guide →
θi∗\theta_i^{*}

Symbol theta_i^*

the old optimum for parameter i and λ\lambda sets how strongly the old task is protected.

Read this term in its guide →
ii

Starting index or lower bound: i

This label says where the repeated addition, multiplication, or accumulation starts. Read its value or condition together with the article’s description of the index.

Read this term in its guide →

How to interpret it

Read it with the definitions, units, and assumptions supplied by the article.

Research cited beside this formula

Learn the mathematics behind it

Published contexts (1)

A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.

L(θ)=Lnew(θ)+λ∑iFi(θi−θi∗)2,\mathcal{L}(\theta) = \mathcal{L}_{\mathrm{new}}(\theta) + \lambda \sum_{i} F_i\left(\theta_i - \theta_i^{*}\right)^2,

Equation 12 · Edge AI & Electronics

The Hardest Unsolved Problems in Small and On-Device AI

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

One standard mitigation is regularization: penalize the optimizer for moving parameters that mattered to earlier tasks. The best-known form estimates a per-parameter importance weight — commonly the diagonal of the Fisher information, FiF_i — from the old task, and adds it to the new loss: L(θ)=Lnew(θ)+λ∑iFi(θi−θi∗)2\mathcal{L}(\theta) = \mathcal{L}_{\mathrm{new}}(\theta) + \lambda \sum_{i} F_i\left(\theta_i - \theta_i^{*}\right)^2. where θi∗\theta_i^{*} is the old optimum for parameter i and λ\lambda sets how strongly the old task is protected. This equation exposes the actual trade rather than resolving it: raising λ\lambda protects old knowledge at the direct expense of how much the new update is allowed to change the model, and there is no value of λ\lambda that removes the trade — only one that relocates it. It also…

Meanings in this article

Equation guide → · Article →