$$I_{\mathrm{gemm}} = \frac{2mnk}{s\,(mk + kn + mn)}$$

Equation 6 · AI Hardware & Semiconductors

What an AI Accelerator Actually Is: Silicon, Packaging, and the Memory It Can Reach

This equation states an equality: the expressions on both sides have the same value under the article’s assumptions.

Multiplying an m $\times$ k matrix by a k $\times$ n matrix performs 2mnk floating-point operations while touching mk + kn + mn elements. If each element occupies s bytes, the ratio of work done to bytes moved is $I_{\mathrm{gemm}} = \frac{2mnk}{s\,(mk + kn + mn)}$. which for large square matrices grows in proportion to the dimension. Double the matrix size and you roughly double the work performed per byte fetched. Almost nothing else in general-purpose computing behaves this way, and it is the entire economic basis for building a machine around one operation.

Meanings in this article

• $I_{\mathrm{gemm}}$: the ratio of work done to bytes moved.
• $m$: the number of rows in the first matrix.
• $n$: the number of columns in the second matrix.
• $k$: the shared inner dimension: columns in the first matrix and rows in the second.
• $s$: bytes occupied by each element.

Equation guide → · Article →