Number of requests

The passage around this formula

Write the total lifetime computation of a deployed system as $C_{\mathrm{total}} = C_{\mathrm{pre}} + C_{\mathrm{post}} + Q \cdot \bar{c}_{\mathrm{inf}}$. where $C_{\mathrm{pre}}$ is pretraining compute, $C_{\mathrm{post}}$ is post-training compute, Q is the number of served requests over the system’s life, and $\bar{c}_{\mathrm{inf}}$ is the mean compute per request. Until roughly 2024 the third term was treated as approximately fixed for a given model, and public discussion of capability collapsed onto the first. That assumption no longer…

Read this part in the article →