=

The passage around this formula

and the sustainable average power of an always-on system operating at duty cycle d (the fraction of time spent active rather than in a low-power sleep state) is $P_{\text{avg}} = d \cdot P_{\text{active}} + (1-d) \cdot P_{\text{sleep}}$. This identity is trivial algebraically, but it is exactly the trade a systems designer is making when choosing between “a bigger, slower model that runs less often” and “a smaller, faster model that still meets the same detection latency.” Neither $P_{\text{active}}$ nor $t_{\text{active}}$ is fixed by the model architecture alone; both depend on the accelerator the model runs on.

Read this part in the article →