Symbol c_i
is part of the quantity the equation computes from the expression on the right.
Read this term in its guide →Published equation contexts
Why this formula appears here
Bahdanau, Cho and Bengio proposed letting the decoder search the source for the parts relevant to each output word, rather than reading from a single compressed vector [ 9 ] . The decoder computes, at each output step i , a context vector as a weighted sum of all encoder states: . The capacity of the intermediate representation now grows with the input rather than being fixed in advance. They further reported that the learned alignments corresponded well with human linguistic intuition — an interpretability result that arrived free with a performance fix, which is rare.
Read the representative guide
Symbol j
j occurs above the fraction bar. The numerator is divided by the entire denominator below it.
Read this term in its guide →Symbol T_x
occurs below the fraction bar. The quantity above the bar is divided by this expression; zero is excluded as a denominator.
Read this term in its guide →Symbol alpha_ij
alphj is an input to the expression that computes the quantity on the left.
Read this term in its guide →Symbol h_j
is an input to the expression that computes the quantity on the left.
Read this term in its guide →Symbol e_ij
j occurs above the fraction bar. The numerator is divided by the entire denominator below it.
Read this term in its guide →Symbol k
k occurs below the fraction bar. The quantity above the bar is divided by this expression; zero is excluded as a denominator.
Read this term in its guide →Symbol e_ik
k occurs below the fraction bar. The quantity above the bar is divided by this expression; zero is excluded as a denominator.
Read this term in its guide →Starting index or lower bound: j=1
This label says where the repeated addition, multiplication, or accumulation starts. Read its value or condition together with the article’s description of the index.
Read this term in its guide →Ending index or upper bound: T_x
This label says where the repeated addition, multiplication, or accumulation stops. It sets the last term or end of the range.
Read this term in its guide →Denominator: sum_k=1^T_x exp(e_ik)
The complete quantity below the fraction bar; it must be nonzero for this division.
Read this term in its guide →Starting index or lower bound: k=1
This label says where the repeated addition, multiplication, or accumulation starts. Read its value or condition together with the article’s description of the index.
Read this term in its guide →Ending index or upper bound: T_x
This label says where the repeated addition, multiplication, or accumulation stops. It sets the last term or end of the range.
Read this term in its guide →How to interpret it
With a fixed numerator, increasing a nonzero denominator reduces the fraction. Read it with the definitions, units, and assumptions supplied by the article.
Research cited beside this formula
Published contexts (1)
A symbol can carry a different meaning in another article. Each occurrence keeps its own guide and term definitions.
Equation 5 · Foundation Models
From n-Grams to Reasoning Models: A Technical History of the Language Model
This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.
Bahdanau, Cho and Bengio proposed letting the decoder search the source for the parts relevant to each output word, rather than reading from a single compressed vector [ 9 ] . The decoder computes, at each output step i , a context vector as a weighted sum of all encoder states: . The capacity of the intermediate representation now grows with the input rather than being fixed in advance. They further reported that the learned alignments corresponded well with human linguistic intuition — an interpretability result that arrived free with a performance fix, which is rare.
Equation guide → · Article →