What does this equation mean?

One standard mitigation is regularization: penalize the optimizer for moving parameters that mattered to earlier tasks. The best-known form estimates a per-parameter importance weight — commonly the diagonal of the Fisher information, $F_i$ — from the old task, and adds it to the new loss: $\mathcal{L}(\theta) = \mathcal{L}_{\mathrm{new}}(\theta) + \lambda \sum_{i} F_i\left(\theta_i - \theta_i^{*}\right)^2$. where $\theta_i^{*}$ is the old optimum for parameter i and $\lambda$ sets how strongly the old task is protected. This equation exposes the actual trade rather than resolving it: raising $\lambda$ protects old knowledge at the direct expense of how much the new update is allowed to change the model, and there is no value of $\lambda$ that removes the trade — only one that relocates it. It also exposes a cost specific to on-device deployment: computing and storing $F_i$ for every parameter, and doing so repeatedly as the device keeps learning, is itself memory and compute that a phone-class budget has to find room for, on top of whatever the update itself costs.