OpenAI Model Systems from First Principles: Weights, Post-Training, and Inference Compute

A checkpoint is not a system

Almost every confused argument about frontier language models begins by treating “the model” as a single object with a single capability. It is more useful, and more nearly true, to treat a deployed OpenAI model system as a compute budget allocated across three stages that are related but separately controllable: the compute spent once to fit parameters, the compute spent afterwards to shape behaviour, and the compute spent again on every individual request.

Write the total lifetime computation of a deployed system as

where $C_{\mathrm{pre}}$ is pretraining compute, $C_{\mathrm{post}}$ is post-training compute, $Q$ is the number of served requests over the system’s life, and $\bar{c}_{\mathrm{inf}}$ is the mean compute per request. Until roughly 2024 the third term was treated as approximately fixed for a given model, and public discussion of capability collapsed onto the first. That assumption no longer holds. On current OpenAI models, $\bar{c}_{\mathrm{inf}}$ is a parameter the caller sets: as verified on 8 August 2026, the model guidance documents a reasoning_effort control taking the values none, low, medium, high, xhigh, and max, alongside a separate text.verbosity setting ^[2].

That single API field breaks a habit of thought. “GPT-5.6 scores $x$” is not a statement about a model. It is a statement about a model at a chosen reasoning effort, under a particular prompt, decoding configuration, and evaluation harness. The quantity being reported is a function evaluated at a point, and the point is usually unstated.

This article works through the three stages in order, then asks what can actually be inferred about such a system from the outside.

What pretraining buys, and what it costs

The architecture underneath has been remarkably stable. The decoder-only transformer described in 2017 remains the load-bearing structure: attention over a sequence, position-wise feedforward layers, residual connections ^[4]. What has changed is scale, data, and everything wrapped around it.

Pretraining compute for a dense transformer is well approximated by

with $N$ parameters and $D$ training tokens, the factor of six counting the forward and backward passes. Empirically, test loss falls as a power law in each of parameters, data, and compute over many orders of magnitude, a relationship first characterised systematically by Kaplan and colleagues ^[5]. The important structural feature is the functional form: a term of the shape

has an irreducible floor $L_\infty$ and diminishing returns above it. Doubling $N$ buys a fixed decrement in loss, not a fixed multiple of capability.

Hoffmann and colleagues then showed that the allocation between $N$ and $D$ at fixed $C$ had been wrong in practice: for compute-optimal training, parameters and tokens should scale in roughly equal proportion, and models of the era were substantially undertrained relative to their size ^[6]. That result changed industry practice, but it optimises the wrong objective for a served product. Compute-optimal training minimises loss for a fixed training budget. A commercial system minimises total cost over training and inference, and when $Q$ is very large the third term in $C_{\mathrm{total}}$ dominates. The rational response is to train smaller models for longer than the compute-optimal recipe suggests, because every parameter removed is paid back on every one of $Q$ requests. The existence of small, cheap variants in each OpenAI generation is consistent with exactly this trade, though the specific training recipes are not disclosed.

What pretraining actually produces is a prior: a distribution over continuations that encodes an enormous amount about language, code, and the structure of the world, and essentially nothing about what the operator wants done with it. It is a store of potential, not a behaviour.

Post-training installs behaviour, not knowledge

The step that turns a prior into a usable system is post-training, and it is best understood as shaping a surface rather than adding material. Ouyang and colleagues demonstrated the canonical form: supervised fine-tuning on demonstrations, a reward model fitted to human preference comparisons, then reinforcement learning against that reward model. Their central empirical result deserves restating because it is so often misremembered — a 1.3B-parameter InstructGPT model was preferred by human raters to the 175B GPT-3 model, despite having over a hundred times fewer parameters ^[7]. Post-training does not add knowledge the prior lacks. It changes which of the prior’s capabilities are reliably elicited.

The second wave of post-training targets reasoning specifically, and it is where the field’s recent public evidence is strongest. Chain-of-thought prompting established that eliciting intermediate steps improves performance on multi-step problems, and that the effect appears only above a certain scale ^[8]. The subsequent move was to stop prompting for those steps and start training for them against verifiable outcomes. The clearest peer-reviewed account is the DeepSeek-R1 work published in Nature, which showed that reasoning behaviour can be incentivised through reinforcement learning on verifiable tasks without human-annotated reasoning traces, with self-verification and strategy revision emerging rather than being demonstrated; the reported AIME 2024 average pass@1 for the RL-only variant rose from 15.6% to 77.9% over training ^[10].

That result is from a different laboratory and a different model family, and it must not be read as a description of OpenAI’s method, which is not disclosed. It is cited here for what it establishes generally: that long deliberation is a trainable policy, that verifiable-reward RL is sufficient to induce it, and therefore that a model’s willingness to spend tokens thinking is a property installed in post-training rather than an emergent accident of scale.

This has a consequence that matters for evaluation. Once deliberation is trained, the model’s output length becomes strongly task-dependent and no longer a stable property of the checkpoint. Any cost model built on average token counts observed in one workload will mis-forecast another.

Inference-time compute is the second scaling axis

The third term in $C_{\mathrm{total}}$ is now a control surface rather than a constant, and it obeys its own scaling behaviour.

Consider a task instance with a policy $\pi_\theta$ and a compute budget $b$ spent at inference, whether as sequential deliberation, parallel sampling, or search against a verifier. Expected quality is some $q(b)$ that rises and saturates. Snell and colleagues studied this directly and reported that allocating test-time compute adaptively to the difficulty of the prompt substantially outperforms uniform allocation, and that in some regimes additional inference compute is a more effective use of a marginal FLOP than additional parameters ^[9]. The practically important half of that finding is the first: the optimal $b$ is a function of the instance, not of the model.

The parallel-sampling case makes the shape of the returns explicit. If a single attempt succeeds with probability $p$ and attempts were independent, the probability that at least one of $k$ succeeds is

which is concave in $k$ and saturates quickly. Two caveats destroy any naive extrapolation from it. Attempts from one model on one prompt are strongly correlated, so realised gains fall well below this bound; and $p_k$ is only achievable if something can identify the successful attempt. Without a verifier, extra samples buy candidates, not answers. This is precisely why the reasoning-effort control and the availability of parallel test-time compute are architectural facts about a product rather than mere quality dials — the GPT-5 system card describes a variant that makes use of parallel test-time compute, and states that all models were evaluated at high reasoning effort ^[1].

The economic reframing is the point of this section. A frontier model is no longer a fixed-price capability. It is a capability curve, and the operator chooses where on the curve to sit for each request. Procurement questions of the form “is model A better than model B” are underspecified until both are placed at a matched cost or matched latency point.

The serving system decides what a token costs

Nothing above determines the price. That is set by the inference system, and its constraints are physical rather than statistical.

Autoregressive decoding is dominated not by arithmetic but by memory traffic. Generating one token requires reading the model weights and the accumulated key–value cache from memory; the arithmetic per byte read is low, so decoding is memory-bandwidth bound while the prefill of a long prompt is compute bound. Pope and colleagues formalised this partitioning analysis and showed how latency, throughput, and cost trade off against one another under different sharding strategies for large transformers ^[11]. The consequence is that “cost per token” is not a property of a model. It is a property of a model plus a batching regime, a sharding layout, and a hardware generation.

Three mitigations account for much of the price movement visible from outside. Grouped-query attention reduces the size of the key–value cache by sharing key and value heads across groups of query heads, trading a small quality cost for a large reduction in the memory that must be streamed per token ^[14]. PagedAttention removes the fragmentation and over-reservation inherent in contiguous KV allocation, allowing far higher batch occupancy at the same memory footprint ^[12]. Speculative decoding uses a cheap draft model to propose several tokens which the target model verifies in a single parallel pass, reducing latency without altering the output distribution ^[13].

That last property is worth dwelling on, because it is the cleanest example of a distinction this article is built around. Speculative decoding changes cost and latency while provably leaving the sampled distribution unchanged. Quantisation, batching policy, and cache eviction do not have that guarantee. A provider can therefore reduce prices in two categorically different ways — by improving the serving system, or by changing what is served — and from the outside these are difficult to distinguish. OpenAI’s documentation records substantial price movement within a single generation: as verified on 8 August 2026, the late-July 2026 entries state that GPT-5.6 Terra costs 20% less and GPT-5.6 Luna 80% less than at their introduction ^[3]. That is a vendor pricing disclosure. It is not, by itself, evidence about what changed underneath.

Families, routers, and the disappearance of “the model”

The product surface has moved in the same direction as the technical one. The GPT-5 system card describes not a model but a system: a fast high-throughput model, a deeper reasoning model, and a real-time router deciding between them based on conversation type, complexity, tool needs, and explicit user intent, with distinct API-exposed variants beneath ^[1]. Current guidance continues the pattern with a named family — gpt-5.6-sol for frontier capability, gpt-5.6-terra for a balance of intelligence and cost, gpt-5.6-luna for efficient high-volume work — and an unsuffixed gpt-5.6 alias that routes to Sol ^[2]. The cadence is rapid: OpenAI’s own change log records GPT-5.4 in March 2026, GPT-5.5 in April 2026, and GPT-5.6 in July 2026 ^[3].

For a user of an alias, three things can change without any action on their part: which member of the family answers, at what reasoning effort, and on what serving stack. For an integrator, this is the single strongest argument for pinning explicit model identifiers and explicit effort levels in production, and for treating an unpinned alias as a convenience for exploration rather than a dependency.

Why these systems are only weakly identifiable

Collect the moving parts and the measurement problem becomes plain. An observed score $s$ is produced by

with weights $\theta$, reasoning effort $e$, prompt and decoding configuration $\mathcal{P}$, evaluation harness $\mathcal{H}$, serving stack $\mathcal{S}$, and date $t$. A published number fixes $s$ and usually reports only part of $\theta$ and $t$. The system is underdetermined: many configurations produce the same score, and the same configuration produces different scores on different dates.

Four rules follow, and they are the practical content of everything above.

Report the effort level, always. A comparison at unstated reasoning effort is uninterpretable, because effort is a free parameter that moves scores substantially and moves cost more.

Compare at matched cost or matched latency, not matched name. Since capability is a curve in $b$, the honest comparison places two systems at equal spend and asks which produces more, or fixes a quality bar and asks which reaches it for less.

Pin identifiers and re-measure on a schedule. An alias is a routing decision made by someone else. A pinned identifier plus a dated re-run is the only way to separate a regression in your prompt from a change in the system.

Distinguish price changes from capability changes. They are announced in the same channel and have different implications; only the second should move an architecture decision.

Predictions, with the observations that would falsify them

These are forecasts, clearly separated from the sourced analysis above. Horizon: 8 August 2028.

One. Per-request compute controls will become more granular rather than fewer, with budgets expressed in units the caller can price — token or time budgets — rather than ordinal labels alone. Disconfirmed if the major providers converge on a single automatic setting with no caller-visible budget control.

Two. The gap between a frontier model at maximum effort and a small model at maximum effort will narrow more slowly than the gap between the same frontier model at maximum and at minimum effort. Put plainly: where you sit on the curve will matter more than which model you chose. Disconfirmed if published matched-cost comparisons show model choice dominating effort choice on general task suites.

Three. Provider-reported evaluation numbers will increasingly be accompanied by the inference budget used, because comparisons without it will have become indefensible. Disconfirmed if leading system cards in 2028 still report headline scores with no budget disclosure.

Four. Serving-side improvements, not parameter growth, will account for the majority of price reduction within a generation. Disconfirmed if published pricing falls chiefly at generation boundaries where new weights ship, rather than mid-generation.

None of these requires a capability discontinuity. They follow from the structure already visible: three separable compute budgets, one of which is now under the caller’s control, served by a system whose costs are governed by memory bandwidth.

What to take away

A frontier model is a store of capability fitted once, a working surface shaped afterwards, and a gate opened by a different amount for every job. The first stage is priced by a scaling law with diminishing returns. The second determines what the store actually produces and is where reasoning behaviour is installed. The third has become a variable the caller sets, which means capability is a function rather than a constant and every benchmark number is a function evaluated at an unreported point.

Treat “the model” as shorthand, never as the unit of analysis. Ask which member of the family, at what effort, on what date, at what cost — and if any of those four is missing, the claim in front of you is not yet a measurement.