Equation 4 · The Hardest Unsolved Problems in Open-Weight AI
What does this equation mean?
Read the formula alongside the article passage below. Each part has a deeper page with its role in the equation, the supporting passage and nearby citations.
This equation states an equality: the expressions on both sides have the same value under the article’s assumptions. Read the equation part by part below; each part has a contextual explanation and a link to its mathematical background.
Read it piece by piece
Symbol P
P is part of the quantity the equation computes from the expression on the right.
Symbol q^N
is one of the signed contributions combined to compute the quantity on the left.
=
The expressions on both sides represent the same quantity under the stated assumptions.
See an illustrated explanation →subtraction
Subtract the following term or group from the preceding one. A leading minus marks a negative quantity.
superscript
A raised number can be a power. When it is a label or bound, it selects a case or the upper limit of a sum; the formula’s structure distinguishes these uses.
See an illustrated explanation →How to interpret it
Read it with the definitions, units, and assumptions supplied by the article.
What the article says around this equation
Put the two facts together and the shape of the unsolved problem is precise. It is not “open weights are dangerous” — that is a separate, contested claim examined below. It is that the field has no method, proposed or implemented, for making a release reversible , and every mitigation on offer — better licences, use-restriction clauses, staged access — governs the decision to release, not anything that happens afterward. A simple way to see why after-the-fact governance cannot rescue the situation is to notice how quickly the number of independent copies dominates any per-copy probability of control. If N copies exist and each copyholder independently retains theirs with probability q close…
Read the full surrounding passage
Put the two facts together and the shape of the unsolved problem is precise. It is not “open weights are dangerous” — that is a separate, contested claim examined below. It is that the field has no method, proposed or implemented, for making a release reversible , and every mitigation on offer — better licences, use-restriction clauses, staged access — governs the decision to release, not anything that happens afterward. A simple way to see why after-the-fact governance cannot rescue the situation is to notice how quickly the number of independent copies dominates any per-copy probability of control. If N copies exist and each copyholder independently retains theirs with probability q close to one, the probability that every single copy is eventually deleted or brought back under control is , so the probability that at least one copy survives indefinitely is . This is a deliberately simplified model — real retention decisions are not independent, and q is not a single fixed number — but it isolates the right variable. Because copying an already-downloaded checkpoint costs approximately nothing, N grows with every re-share, and P() climbs toward one almost as soon as distribution starts, for any q short of certainty. This is a restatement, in one line of algebra, of what NTIA’s 2024 report and RAND’s 2024 report both conclude in prose: monitoring and mitigation are the available tools, and recall is not among them [ 2 , 3 ] .
Sources cited in the article section
- [15] Runaway LLaMA: How Meta's LLaMA NLP Model Leaked ↗
- [3] Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models ↗
- [1] Our Position on Open-Weights Models ↗
- [5] LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B ↗
These citations give research context. Read each source to check which claims it supports.
Return to The Hardest Unsolved Problems in Open-Weight AI