Symbol P_expert

The passage around this formula

…each token to one shared expert plus a fixed number of routed experts, the active parameter count per token is approximately $P_{\mathrm{active}} \approx P_{\mathrm{shared}} + k \cdot P_{\mathrm{expert}}$. where $P_{\mathrm{shared}}$ is the always-active shared-expert capacity, $P_{\mathrm{expert}}$ is the parameter count of one routed expert, and k is the number of routed experts activated per token. Total capacity scales with the number of experts E available to the router, roughly $P_{\mathrm{total}}$ $\approx$ $P_{\mathrm{shared}}$ + E $\cdot$ $P_{\mathrm{expert}}$ , while…

Read this part in the article →