What does this equation mean?

for a dense model with $N_{\mathrm{dense}}$ parameters compared against an MoE model activating $N_{\mathrm{active}}$ of its $N_{\mathrm{total}}$ parameters per token. What this ratio hides is exactly what a compute-only comparison always hides: memory. Serving an MoE model requires holding all $N_{\mathrm{total}}$ parameters resident — on one device or, more often, sharded across several with communication between them whenever a batch’s tokens route to different experts — even though only $N_{\mathrm{active}}$ of them do arithmetic on any given token. A dense model of the same activated size carries no such requirement. The compute saving is real and is the reason MoE models can be commercially attractive to serve; the memory and networking cost it trades against is equally real, and neither the Switch Transformer paper, the Mixtral paper nor the DeepSeek-V3 report discloses enough about any specific production serving stack to say where the net trade lands for a given deployment.