Symbol h
h is part of the quantity the equation computes from the expression on the right.
Read this term in its guide →Published equation contexts
Why this formula appears here
with the forward pass computing h = x + W x against the frozen base weight . The method’s authors report reducing the number of trainable parameters by roughly ten thousand times and GPU memory requirements by roughly three times relative to full fine-tuning of a 175-billion-parameter model, while matching or exceeding full fine-tuning quality and adding no additional inference latency, since the low-rank update can be merged back into the base weight at deployment time [ 1 ] . QLoRA extends the same idea to quantized base weights, and its authors report finetuning a 65-billion-parameter model on a single 48-gigabyte GPU while preserving full 16-bit finetuning performance,…
Read the representative guide
Symbol W_0
is one of the signed contributions combined to compute the quantity on the left.
Read this term in its guide →Symbol x
x is one of the signed contributions combined to compute the quantity on the left.
Read this term in its guide →Symbol Δ W
Δ W is one of the signed contributions combined to compute the quantity on the left.
Read this term in its guide →How to interpret it
Read it with the definitions, units, and assumptions supplied by the article.
Research cited beside this formula
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 2 · Foundation Models
Building a Multimodal AI Application That Actually Uses Its Inputs
This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.
with the forward pass computing h = x + W x against the frozen base weight . The method’s authors report reducing the number of trainable parameters by roughly ten thousand times and GPU memory requirements by roughly three times relative to full fine-tuning of a 175-billion-parameter model, while matching or exceeding full fine-tuning quality and adding no additional inference latency, since the low-rank update can be merged back into the base weight at deployment time [ 1 ] . QLoRA extends the same idea to quantized base weights, and its authors report finetuning a 65-billion-parameter model on a single 48-gigabyte GPU while preserving full 16-bit finetuning performance,…
Equation guide → · Article →