≈

The passage around this formula

Consider an idealised r $\times$ c array holding one operand stationary, with a second operand streamed through in a sequence of length L — for instance, L activation vectors passed through a weight-stationary array of r input rows and c output columns. Before the first result emerges, the data has to propagate across the array’s diagonal, costing roughly r + c - 2 cycles; after the last input enters, the same number of cycles is needed to drain the pipeline. Total latency is therefore approximately $T \approx (r + c - 2) + L$. while the useful work performed is r $\cdot$ c $\cdot$ L multiply-accumulates — one per cell, once per streamed step, in steady state. Dividing useful work by the total cell-cycles…

Read this part in the article →