Equation 2 · Part 7 · Building a Multimodal AI Application That Actually Uses Its Inputs
change
change
What this part means
Capital delta attached to a quantity marks a difference between two values of that quantity; the article’s sign convention determines the order.
Its job in the formula
Capital delta attached to a quantity marks a difference between two values of that quantity; the article’s sign convention determines the order.
Full expression→change→Article meaning
The passage around this formula
with the forward pass computing h = x + W x against the frozen base weight . The method’s authors report reducing the number of trainable parameters by roughly ten thousand times and GPU memory requirements by roughly three times relative to full fine-tuning of a 175-billion-parameter model, while matching or exceeding full fine-tuning quality and adding no additional inference latency, since the low-rank update can be merged back into the base weight at deployment time [ 1 ] . QLoRA extends the same idea to quantized base weights, and its authors report finetuning a 65-billion-parameter model on a single 48-gigabyte GPU while preserving full 16-bit finetuning performance,…
Learn the underlying idea
Addition combines quantities; subtraction measures the signed difference between them. Parentheses show what is combined before the rest of the expression is evaluated.
Open the illustrated addition and subtraction in an equation guide →
Sources cited in the surrounding passage
- [1] LoRA: Low-Rank Adaptation of Large Language Models ↗
- [2] QLoRA: Efficient Finetuning of Quantized LLMs ↗
- [10] PEFT: Parameter-Efficient Fine-Tuning ↗
These citations provide research context; check each source for the exact claim it supports.