addition

The passage around this formula

Multiplying an m $\times$ k matrix by a k $\times$ n matrix performs 2mnk floating-point operations while touching mk + kn + mn elements. If each element occupies s bytes, the ratio of work done to bytes moved is $I_{\mathrm{gemm}} = \frac{2mnk}{s\,(mk + kn + mn)}$. which for large square matrices grows in proportion to the dimension. Double the matrix size and you roughly double the work performed per byte fetched. Almost nothing else in general-purpose computing behaves this way, and it is the entire economic basis for building a machine around one operation.

Read this part in the article →