Equation 11 · Part 1 · The Hardest Unsolved Problems in Small and On-Device AI
Symbol F_i
What this part means
the computing and storing.
Its job in the formula
is a part of this expression. Its role is fixed by the surrounding article and by the operations shown in the formula.
Full expression→Symbol F_i→Article meaning
Where the article explains it
It also exposes a cost specific to on-device deployment: computing and storing for every parameter, and doing so repeatedly as the device keeps learning, is itself memory and compute that a phone-class budget has to find room for, on top of whatever the update itself costs.
The passage around this formula
One standard mitigation is regularization: penalize the optimizer for moving parameters that mattered to earlier tasks. The best-known form estimates a per-parameter importance weight — commonly the diagonal of the Fisher information, — from the old task, and adds it to the new loss:
Learn the underlying idea
A subscript is a label attached below a symbol. It often selects a time step, component, category, or member of a sequence.
Open the illustrated subscripts: which member of a family? guide →
See this notation across published equations →
Sources cited in the article section
- [9] Continual Learning of Large Language Models: A Comprehensive Survey ↗
- [10] Online Continual Learning for Embedded Devices ↗
- [13] On-Device Language Models: A Comprehensive Review ↗
- [11] Catastrophic Forgetting in LLMs: A Comparative Analysis Across Language Tasks ↗
These citations provide research context; check each source for the exact claim it supports.