What does this equation mean?

Low-Rank Adaptation, LoRA, takes a different approach: it freezes the pretrained weight matrix entirely and represents the update as the product of two much smaller matrices. For a frozen weight matrix $W_0$ $\in$ $\mathbb{R}^{d \times k}$ , LoRA represents the adapted weight as $W = W_0 + \Delta W = W_0 + BA, \qquad B \in \mathbb{R}^{d \times r},\ A \in \mathbb{R}^{r \times k},\ r \ll \min(d, k)$. Only B and A are trained; $W_0$ never moves during fine-tuning [ 5 ] . The effect on trainable parameter count for that one matrix is to fall from dk to r(d + k) , which is small whenever the chosen rank r is small relative to d and k — and because BA can be merged back into $W_0$ after training, LoRA adds no extra inference latency once deployed. Hu and colleagues report, for their comparison against full fine-tuning of GPT-3 175B, up to a 10,000-fold reduction in trainable parameters and a threefold reduction in GPU memory requirement, with quality on par with or better than full fine-tuning on the benchmarks they tested [ 5 ] . Those figures are the paper’s own reported comparison for a specific model and are not a guaranteed ratio for every architecture or rank choice, but the underlying mechanism — freezing the base and training a small low-rank update — is what makes LoRA the practical default for narrower behavioral adjustments: the adapter is small enough to store, version, and swap independently of the frozen base, and several task-specific adapters can share one base checkpoint in production rather than each requiring a full duplicate of the model.