What does this equation mean?

Only B and A are trained; $W_0$ never moves during fine-tuning [ 5 ] . The effect on trainable parameter count for that one matrix is to fall from dk to r(d + k) , which is small whenever the chosen rank r is small relative to d and k — and because BA can be merged back into $W_0$ after training, LoRA adds no extra inference latency once deployed. Hu and colleagues report, for their comparison against full fine-tuning of GPT-3 175B, up to a 10,000-fold reduction in trainable parameters and a threefold reduction in GPU memory requirement, with quality on par with or better than full fine-tuning on the benchmarks they tested [ 5 ] . Those figures are the paper’s own reported comparison for a specific model and are not a guaranteed ratio for every architecture or rank choice, but the underlying mechanism — freezing the base and training a small low-rank update — is what makes LoRA the practical default for narrower behavioral adjustments: the adapter is small enough to store, version, and swap independently of the frozen base, and several task-specific adapters can share one base checkpoint in production rather than each requiring a full duplicate of the model.