Actually Deploying an Open-Weight Model in Production

Four decisions before the first fine-tuning run

“Open weight” gets treated in most engineering conversations as a synonym for “no further obligations” — download the file, load it, ship it. That reading survives exactly until a legal team, a security reviewer, or a regression in production output asks a question the download step never answered. A team taking an open-weight model like Llama into a commercial product is actually making at least four separate decisions, each of which has a documented answer that can be checked against a primary source rather than assumed from how the last project went.

The first is whether this specific deployment, at this specific scale and for this specific purpose, sits inside what Meta’s licence and use policy actually grant — not what “open” implies in general, but what the clauses say. The second is which released checkpoint to start adapting: Meta does not ship one Llama per size, it ships at least two, and the choice is a design decision with real consequences, not a default. The third is how to adapt it — full-parameter fine-tuning, a low-rank adapter, or a quantized variant of one — each with documented compute costs and documented failure modes, not equally good answers to every problem. The fourth is whether the model that comes out the other end of that process is still as safe as the one Meta shipped, and how the team would know if it were not.

This article works through those four in order, grounding every recommendation in Meta’s own licence text, its own model cards and fine-tuning documentation, and the peer-reviewed papers that establish the fine-tuning methods and their risks. None of this is exotic. All of it is skipped often enough that it is worth writing down as a checklist rather than a set of assumptions.

What the licence actually binds you to

The Llama 4 Community License Agreement grants a broad set of rights on its face: a non-exclusive, worldwide, non-transferable, royalty-free licence to use, reproduce, distribute, and modify the Llama materials, and to create derivative works from them ^[1]. Read only that far, and “open” looks unconditional. It is not, and the two carve-backs are the part of the document a production team actually has to act on.

The first is a user-scale threshold. The agreement states that if, on the Llama 4 version release date, the products or services made available by the licensee or its affiliates had more than 700 million monthly active users in the preceding calendar month, the licensee must request a separate licence from Meta, which Meta may grant or withhold at its discretion ^[1]. The detail practitioners most often miss is the anchor: the threshold is pegged to a specific version’s release date, not to the day a team happens to check it. A company that was well under the threshold when it first integrated Llama 3.1 does not get to assume that clearance carries forward automatically to Llama 4, or to whatever generation follows it — each version’s licence re-anchors the check to its own release date, which means the compliance step that actually matters is dating the check, not performing it once and filing the result away.

The second carve-back is a set of naming and attribution obligations, and they attach to different actions. Redistributing Llama materials, or a product built on them, requires prominently displaying “Built with Llama” on a related website, user interface, blog post, about page, or product documentation, and retaining the required notice text in any copy that is distributed ^[1]. Creating an AI model that is itself built on Llama materials and then distributing or making that derivative model available carries a narrower and stricter obligation on top of that: the derivative model’s name must begin with “Llama” ^[1]. A team fine-tuning Llama for internal use only, with no distributed model artifact, is in a different position under the licence than a team shipping the fine-tuned checkpoint itself — the naming rule binds the second case specifically, the attribution display requirement is broader.

Layered on top of the licence, and incorporated into it by reference, is the Acceptable Use Policy, which is where the field-of-use restriction actually lives. It prohibits four broad categories: violating law or others’ rights, including child exploitation, human trafficking, harassment, and infringing third-party rights; activities presenting a risk of death or bodily harm, explicitly naming military and warfare use, nuclear applications, espionage, weapons development, and operation of critical infrastructure; deception and misinformation, covering fraud, impersonation, and falsely presenting model output as human-generated; and a residual category covering failure to disclose known risks to end users and interacting with third-party tools built for unlawful content ^[2]. Violations can be reported through several separate channels Meta maintains — a GitHub issue tracker for model problems, a dedicated feedback form for risky output, a security-disclosure page, and an email address for policy violations specifically ^[2] — which is itself informative: an incorporated-by-reference policy with a live reporting mechanism is a document Meta can and does revise, not a fixed text a team reads once and never revisits.

None of this makes Llama’s licence unusual among model releases that describe themselves as open. It does make “we checked the licence” an insufficient sentence in a compliance review. A defensible check has four parts: recording the licensee’s monthly active user count against the specific version’s release date, not against the date of the review; mapping the product’s actual and planned use cases against the Acceptable Use Policy’s categories explicitly, including any new use a fine-tuned variant is built for, since a narrow fine-tune for a new purpose can cross into a newly prohibited category even when the base deployment never did; satisfying the attribution and naming requirements as currently written for the version in use, since the exact notice text changes release to release; and keeping a dated record of all three, because the licence and the policy it incorporates are both revisable documents, and “compliant on the day we checked” is a claim about a point in time.

Base or instruct: a choice, not a default

Meta releases each Llama generation, at each parameter scale, as more than one checkpoint. Llama 4 Scout and Llama 4 Maverick both ship as a base pretrained model and as a separate instruction-tuned model, and Meta’s own release documentation and Hugging Face’s model cards recommend the instruction-tuned variant for most direct use ^{[4, 3]}. The two checkpoints are not the same artifact with a different label. Vendor-published specifications for Llama 4 report a materially longer supported context length for the instruction-tuned releases than the base weights — figures reported include a 10-million-token window for Scout’s instruction-tuned release against a 256,000-token figure for the base weights, and a 1-million-token window for Maverick’s instruction-tuned release ^[4]. Those are the vendor’s own published numbers, not independently reproduced by this article, and should be treated as claims about the released artifact rather than measured guarantees for every deployment context.

The practical question for a team building a fine-tuning pipeline is which of the two checkpoints to start from, and the honest answer is that it depends on how different the target behavior is from what the checkpoint already does. Starting from the base pretrained model makes sense when the intended product installs a genuinely different behavior distribution than a general assistant — a narrow classifier, a structured-output-only interface, or a domain where the chat-formatted refusal patterns baked into the instruction-tuned release would actively fight the training objective. Starting from base means training against an undirected prior with no existing alignment layer to work around, and no existing layer to accidentally damage.

Starting from the instruction-tuned release is the right default for the much more common case: a product that wants the model to keep behaving like an assistant — following instructions, refusing some categories of request, producing formatted output — while adjusting or narrowing that behavior rather than replacing it wholesale. Here, a large share of the useful behavior is already installed by Meta’s own post-training process, and further fine-tuning is incremental rather than foundational, which is both the appeal and the risk: incremental fine-tuning on top of an already-aligned checkpoint is exactly the setting where existing safety behavior can erode without anyone intending it to, a point this article returns to below.

Full fine-tune, LoRA, or QLoRA

Once a starting checkpoint is chosen, the adaptation method determines both what it costs to train and what has to be stored, audited, and shipped afterward. Three approaches dominate current practice, and each has documented tradeoffs rather than a single “best” answer.

Full-parameter fine-tuning updates every weight in the model and is memory-bound in a way that scales directly with model size. Meta’s own fine-tuning documentation states plainly that an 8-billion-parameter Llama model will not fit into a single GPU for full fine-tuning, because training has to hold the parameters, the gradients, and the optimizer states in memory simultaneously, not just the parameters alone ^[7]. The standard mitigation for multi-GPU full fine-tuning is fully sharded data parallelism, which shards those states across devices rather than replicating them, so that a larger model fits within the same aggregate memory budget than would fit under simple data parallelism on the same hardware ^[7]. Full fine-tuning is the right tool when a task requires a broad rewrite of behavior across many layers and the team has the accelerator budget and multi-GPU infrastructure to support it.

Low-Rank Adaptation, LoRA, takes a different approach: it freezes the pretrained weight matrix entirely and represents the update as the product of two much smaller matrices. For a frozen weight matrix $W_0 \in \mathbb{R}^{d \times k}$, LoRA represents the adapted weight as

Only $B$ and $A$ are trained; $W_0$ never moves during fine-tuning ^[5]. The effect on trainable parameter count for that one matrix is to fall from $dk$ to $r(d + k)$, which is small whenever the chosen rank $r$ is small relative to $d$ and $k$ — and because $BA$ can be merged back into $W_0$ after training, LoRA adds no extra inference latency once deployed. Hu and colleagues report, for their comparison against full fine-tuning of GPT-3 175B, up to a 10,000-fold reduction in trainable parameters and a threefold reduction in GPU memory requirement, with quality on par with or better than full fine-tuning on the benchmarks they tested ^[5]. Those figures are the paper’s own reported comparison for a specific model and are not a guaranteed ratio for every architecture or rank choice, but the underlying mechanism — freezing the base and training a small low-rank update — is what makes LoRA the practical default for narrower behavioral adjustments: the adapter is small enough to store, version, and swap independently of the frozen base, and several task-specific adapters can share one base checkpoint in production rather than each requiring a full duplicate of the model.

QLoRA extends this further for the specific case where the base checkpoint itself does not fit in available accelerator memory at full precision. It first quantizes the frozen base model to a 4-bit format its authors call NormalFloat, described as information-theoretically optimal for normally distributed weights; it quantizes the quantization constants themselves in a second pass to shrink their footprint further; and it uses paged optimizers to absorb memory spikes from long sequences during training ^[6]. Backpropagation runs through the frozen 4-bit base into 16-bit LoRA adapters trained on top of it. Dettmers and colleagues report finetuning a 65-billion-parameter model on a single 48 GB GPU while preserving performance comparable to full 16-bit fine-tuning on the benchmarks tested, with their best resulting model reaching what they report as 99.3% of a reference chat model’s rated quality on the Vicuna benchmark after 24 hours of fine-tuning on one GPU ^[6]. That is a specific, vendor-reported benchmark result on one evaluation, not a general claim that quantized adaptation is indistinguishable from full-precision fine-tuning on every task — the paper’s own contribution is evidence that the gap can be made small on the benchmarks it measured, which is a narrower claim than “the gap is always small,” and is exactly why the fine-tuned result still needs to be evaluated on the target task rather than assumed equivalent.

Meta’s own fine-tuning guidance recommends parameter-efficient methods specifically to reduce hardware requirements and to limit catastrophic forgetting relative to full fine-tuning ^[7], which lines up with the tradeoff practitioners actually face: full fine-tuning for a broad behavioral rewrite with the budget to support it; LoRA as the default for a narrower adjustment layered on existing capability; QLoRA specifically when the unquantized base does not fit in the accelerator memory available at all.

Testing whether the fine-tuned model got less safe

Every adaptation method above changes the model’s weights, and changed weights can change refusal behavior in ways that were never the training objective. Qi and colleagues showed this directly: fine-tuning an aligned language model on roughly ten adversarially chosen examples, at a cost under twenty US cents through a hosted fine-tuning API, was sufficient to compromise the safety guardrails of GPT-3.5 Turbo in their tests ^[8]. That specific result was demonstrated against a closed model reached through an API rather than against Llama directly, and should be attributed precisely as that — but the mechanism it demonstrates, that gradient updates during fine-tuning can reshape refusal behavior as a side effect, is general to fine-tuning aligned language models, which is exactly why it matters for anyone adapting Llama’s instruction-tuned release. The more surprising half of the same paper’s findings is that this is not only an adversarial-data problem: fine-tuning on entirely benign, commonly used datasets also measurably degraded safety alignment in their tests, though to a lesser extent than the deliberate attack ^[8]. A team with no intention of weakening the model’s refusal behavior can still weaken it simply by fine-tuning on ordinary task data.

Meta’s own answer to this problem, at the vendor’s own layer, is Llama Guard — a purpose-built classifier rather than a property expected to persist automatically in the fine-tuned model itself. Llama Guard 4 is a 12-billion-parameter dense safety classifier pruned from the Llama 4 Scout pretrained model, built to classify both prompts and model responses against a 14-category hazard taxonomy aligned to the MLCommons standard, and reported by Meta to improve recall over the prior Llama Guard 3 generation on English-language content while holding down its false-positive rate ^[9]. Used as an independent evaluator rather than as part of the fine-tuning run itself, it gives a practical structure for a before-and-after safety check: hold out a fixed refusal and redirection benchmark set that the fine-tuning data never touches; run that set through both the pre-fine-tune and post-fine-tune checkpoints; score both sets of responses with Llama Guard 4 or an equivalent classifier that was not itself fine-tuned alongside the target model, since a classifier trained in the same run shares the target model’s blind spots rather than checking them independently; and treat any category where the post-fine-tune safe-response rate drops as a regression requiring investigation before release, not as noise to average away — Qi and colleagues’ own result shows how small an intervention can be sufficient to produce a large, targeted shift in exactly this kind of category-level statistic ^[8].

This check matters more, not less, when the starting checkpoint was the instruction-tuned release rather than the base model, because the instruction-tuned release is the one that carries refusal and safety behavior in the first place. Fine-tuning the base pretrained model for a narrow task starts from a checkpoint with no such installed behavior to regress on that axis — though it correspondingly has no such behavior to rely on either, and any safety properties the product needs have to be established and evaluated on their own terms rather than assumed inherited.

Provenance and versioning in a commercial product

A commercial product built on a fine-tuned derivative of an open-weight model needs to answer, reliably and months or years after the fact, a small set of specific questions — and answering them requires having recorded the facts at the time, not reconstructing them later from memory or from whichever engineer happens to still be on the team.

Which exact checkpoint was the starting point matters concretely, because “Llama” names a family, not a single artifact: Meta ships base and instruction-tuned variants at multiple parameter scales within each generation ^{[3, 4]}, and a provenance record that says only “Llama 4” without specifying which released checkpoint has not actually recorded what the product was built on. Which licence version governed the moment the checkpoint was accepted matters because obligations like the user-scale threshold are anchored to a specific version’s release date ^[1] — a product’s compliance posture can turn on a date fact that has to be written down, not remembered. Which fine-tuning method, hyperparameters, and training data were used matters because it is exactly what a later safety re-evaluation would need to reproduce or investigate. And which safety evaluation result gated the release, against which held-out test set and which version of the evaluating classifier, matters because Llama Guard itself is a versioned, updated artifact ^[9] — an evaluation result is only interpretable alongside the version of the evaluator that produced it.

This is not a novel demand invented for open-weight deployment. Mitchell and colleagues proposed structured model documentation nearly a decade before Llama’s release, arguing that a released model should be accompanied by a card recording its intended use, its evaluation data and procedure, and its caveats and recommendations, precisely so a downstream user is not left reconstructing that information from institutional memory ^[10]. Their original framing centered on fairness and demographic evaluation, but the discipline generalizes directly to a commercial product built on a modified open-weight checkpoint: a provenance record is only as useful as its willingness to be checked, and it should be version-controlled alongside the artifact it describes rather than treated as documentation to write after the fact if a reviewer happens to ask. Meta follows this same discipline for its own releases — Llama Guard’s model card is itself a versioned, checkable document ^[9] — which is the clearest evidence that this is a standard the model’s own provider treats as load-bearing, not optional, for exactly the kind of artifact a downstream team is now building on top of.

A practical minimum for a commercial deployment’s provenance record follows from those four questions: the exact checkpoint identifier and its published integrity hash where Meta provides one; the licence version and its effective release date; the fine-tuning method, code revision, and training data used; and the held-out evaluation set, classifier version, and date used to clear the result for release. Five facts, each independently checkable against a primary source, and each one already treated as the load-bearing content of a model card by the organization that released the base checkpoint in the first place ^[10].

Predictions, with the observations that would falsify them

These are forecasts, clearly separated from the sourced analysis above. Horizon: 15 August 2028. They assume no major jurisdiction bans open-weight fine-tuning outright and that Meta continues releasing at least one further Llama generation in that window.

One. Safety-regression testing will move from a recommended practice into an explicit, documented step inside vendor fine-tuning guides themselves, rather than remaining something practitioners are expected to infer from safety research. Indicator: whether Meta’s own fine-tuning documentation, or an equivalent walkthrough from another major open-weight releaser, includes an explicit pre- and post-fine-tune safety-evaluation step in its default recipe. Disconfirmed if major vendor fine-tuning guides in 2028 still ship without any safety-regression step in the standard walkthrough.

Two. User-scale thresholds of the kind Llama’s licence uses will persist across major open-weight releases rather than being replaced by unconditional grants, because the structure has become an accepted middle position among large releasers, not because it is uncontested. Indicator: whether newly released frontier open-weight licences in 2028 include a monthly-active-user-style or revenue-style threshold. Disconfirmed if three or more of the largest weight releasers ship fully unconditional, Apache- or MIT-style licences for frontier-scale models by that date.

Three. Quantized adapter methods in the QLoRA family will become the default entry point in published fine-tuning tutorials for frontier-scale open-weight models, with full-parameter fine-tuning positioned as the exception for teams with dedicated multi-GPU budgets rather than the default path. Indicator: the proportion of vendor and community fine-tuning cookbooks that default to a quantized adapter method. Disconfirmed if full-parameter fine-tuning remains the default recommended path in major vendor cookbooks for frontier-scale models.

The checklist, not the download

None of the four decisions this article has worked through is a technicality, and none of them is answered by the act of downloading the weights. The licence and its incorporated use policy specify who needs a separate agreement with Meta and what the product may not be used for, dated to a specific version’s release. The choice between a base and an instruction-tuned checkpoint sets how much existing behavior the fine-tuning process is working with rather than working around. The choice of adaptation method sets what the training run costs, what has to be stored and versioned afterward, and how much of the frozen base’s precision survives into the merged result. And the safety evaluation is the only mechanism that actually confirms the model leaving the build bench still refuses what it refused before — the fine-tuning process does not guarantee that on its own, in a fine-tuning framework’s documentation, or in a licence’s text.

Each of those four has a source a team can point to, quote, and re-check on a schedule: a licence and use policy with dated obligations, a model card describing exactly which checkpoint was released, a paper reporting exactly what a fine-tuning method costs and what it achieved on stated benchmarks, and a classifier with its own versioned model card standing in for the question “did this get less safe.” Treating any of the four as self-evident from the fact of the model being open is the gap between a deployment that can answer questions eighteen months later and one that cannot.