There is no second channel
Every vulnerability in the injection class has the same shape. A system receives something it should have treated as inert and instead executes it: a quoted string that becomes a clause of a query, a comment field that becomes a script, a filename that becomes a shell command. And in every one of those cases, the durable fix was structural rather than statistical. Parameterised queries do not detect malicious SQL; they carry the query text and the parameter values on separate channels, so no arrangement of characters inside a parameter can become part of the statement. Non-executable memory pages do not identify shellcode; they mark a page as writable or executable and never both. The pattern is consistent: build a boundary the attacker’s content cannot cross, and stop trying to inspect the content.
That move is not available inside a language model. This is the fact from which everything else in this article follows, and it is worth being precise about why.
A transformer consumes one sequence of tokens. The system prompt, the developer’s tool definitions, the user’s message, the contents of a retrieved document and the output of a tool call all arrive as tokens drawn from one vocabulary, laid out along one position axis, attended to by one set of attention heads. The chat roles that appear to separate them are themselves tokens — reserved strings in the same vocabulary, given meaning by a convention the model learned during post-training. There is no privileged bit accompanying a token that marks it as data rather than instruction, because there is no place in the architecture for such a bit to live and no mechanism that would enforce it if there were. The model’s compliance with the system prompt over a conflicting instruction found in a retrieved web page is a learned statistical preference, not a boundary.
This is not a novel or contrarian reading. The OWASP Gen AI Security Project’s 2025 entry for prompt injection states the consequence plainly: “Given the stochastic influence at the heart of the way models work, it is unclear if there are fool-proof methods of prevention for prompt injection” [2]. NIST’s adversarial machine learning taxonomy places the same class of attack inside a formal terminology of attacks and mitigations for practitioners, which is itself an admission of category: things you have a taxonomy and a mitigation catalogue for are things you manage, not things you close [3].
The analytical conclusion, and it is mine rather than any cited document’s: if you cannot separate data from instructions at the input, you must stop trying to defend the input and instead defend what an instruction is able to reach. The rest of this article is an argument for where that boundary goes.
Direct and indirect, and why the second one matters
The OWASP taxonomy divides the attack into two forms. “Direct prompt injections occur when a user’s prompt input directly alters the behavior of the model in unintended or unexpected ways”, and “indirect prompt injections occur when an LLM accepts input from external sources, such as websites or files” [2].
Direct injection is the familiar case and, from an operator’s point of view, the less interesting one. The person typing is the person who bears the consequences; the threat model is largely one of a user pushing the system past its intended function on their own account. It matters for abuse and for policy compliance. It rarely produces a confidentiality breach against a third party.
Indirect injection is a different kind of thing. Greshake and colleagues gave the first systematic account in 2023, and their framing has held up: LLM-integrated applications “blur the line between data and instructions”, which allows adversaries to “remotely (without a direct interface) exploit LLM-integrated applications by strategically injecting prompts into data likely to be retrieved” [1]. They derived a taxonomy from a computer-security perspective covering data theft, worming and information ecosystem contamination, and demonstrated the attacks against real systems of the day including a search chat product and code-completion engines. Their summary of the mechanism is the load-bearing sentence: processing retrieved prompts “can act as arbitrary code execution, manipulate the application’s functionality, and control how and if other APIs are called” [1].
Read that as an operator. The attacker does not need an account on your system, does not need to send you anything, and does not need to know you exist. They need only to place text somewhere your system will one day read: a public web page, a code comment in a dependency, a calendar invitation, a support ticket, a shared document, a product review, the alt text of an image, the body of an email that your assistant is asked to summarise. The reach of the attack is the reach of your retrieval.
The measurement side of this has matured. AgentDojo, a dynamic evaluation environment for prompt injection against tool-using agents, was built precisely because static test sets do not capture agents that act. It comprises 97 realistic tasks spanning domains such as email management, e-banking and travel booking, together with 629 security test cases, and its authors report that existing attacks break some but not all security properties, while models struggle with many of the tasks even in the absence of any attack [7]. Two things follow. First, there is now a shared instrument, which means claims about robustness can be compared rather than merely asserted. Second, the reported numbers are not zero, and no serious party claims they are.
The deputy holds your keys
The right classical frame for this is the confused deputy, and it is worth stating carefully because it relocates the vulnerability away from the model and onto the credential.
The Cloud Security Alliance’s research note puts the classical definition and the modern instance together: the confused deputy problem is “a classical access control vulnerability in which a privileged program is tricked by a less-privileged caller into misusing its authority”, and it “has re-emerged as a high-severity threat pattern in AI agent deployments” [5]. Where an agent holds broad credentials to act for a user, any input channel it processes becomes an attack surface, and injected instructions are carried out with the agent’s full authority even though the operator authorised nothing of the kind [5].
Notice what is not happening. There is no memory corruption. No authentication is bypassed. No token is stolen. Every authorisation check in the system passes, and passes correctly, because each one is being asked the wrong question. The service asks “is this caller permitted to perform this action” and the answer is yes, because the caller is the application and the application is permitted. Nothing in the request encodes the fact that the reason for the action originated in a document written by a stranger. The authority is genuine; only the intent behind its use is foreign.
Saltzer and Schroeder named the counter-principle fifty years ago, in the paper that also gave us fail-safe defaults, complete mediation and separation of privilege: “Every program and every user of the system should operate using the least set of privileges necessary to complete the job” [4]. That is not a platitude here. It is the specific control that determines the blast radius of an injection, because the injection inherits exactly the privilege set the deputy carries and not one permission more.
Protocol designers have begun to encode this. The Model Context Protocol specification’s security best practices document devotes a named section to the confused deputy problem in proxy servers, and a separate section to scope minimisation, whose risk list reads as a compact statement of the whole problem: expanded blast radius from a broad token, privilege chaining that lets an attacker reach high-risk tools without any further elevation, and audit noise in which “single omnibus scope masks user intent per operation” [6]. Its prescribed model is progressive: a minimal initial scope covering low-risk discovery and read operations, with incremental elevation challenged at the point a privileged operation is first attempted [6].
Where the data leaves
An injected instruction that cannot reach anything is a curiosity. The channels that turn it into a breach are the egress paths, and the recurring surprise for teams is how many of them were never classified as security-relevant capabilities at all.
The first category is rendered content. If the surface that displays the model’s output will fetch a remote resource in the course of rendering — an image reference, a link preview, an embedded asset — then the act of rendering performs an outbound request to a location the model chose, and any information the model can place into that location’s path or query string leaves the trust boundary. No click is required, and often no visible artefact appears. This channel needs very little capacity to be catastrophic: credentials, tokens and record identifiers are small.
The second is tool-mediated egress. Anything the agent can invoke that emits data outward is an exfiltration path, whether or not it was designed as one: sending mail, posting a comment, opening a pull request, writing to a shared drive, issuing an HTTP request, resolving a hostname. Greshake and colleagues identified precisely this in noting that injected content can “control how and if other APIs are called” [1]. Simon Willison’s widely adopted shorthand names the dangerous combination: private data, exposure to untrusted content, and “the ability to externally communicate in a way that could be used to steal your data” [18]. The framing’s value is operational — it gives you a question to ask of an architecture diagram that has a yes or no answer.
This is not hypothetical. CVE-2025-32711 is a catalogued vulnerability in Microsoft 365 Copilot, published on 11 June 2025, described by the National Vulnerability Database as “Ai command injection in M365 Copilot allows an unauthorized attacker to disclose information over a network”, classified as CWE-74 and scored 7.5 High by NIST, with the vector indicating network attack vector, no privileges required and, notably, no user interaction required [17]. It is worth recording that Microsoft, as the assigning authority, scored the same issue 9.3 Critical [17]. That two competent organisations differ by nearly two points on the same defect is itself informative: the disagreement is about scope and impact modelling for a class of vulnerability that the existing scoring vocabulary was not designed for, and an operator should treat both numbers as estimates rather than measurements.
The analytical point: output filtering is a weak control against exfiltration because the adversary needs so little bandwidth, and because the channel is frequently a feature of the client rather than of the model. The stronger control is at the network and capability layer — an agent that cannot reach an arbitrary host cannot send anything to one, whatever it has been persuaded to want.
Everything upstream of the context window is input
The supply chain for a model-integrated system is wider than the code, and three of its branches deserve separate treatment.
Model weights. Two results bound the problem. Hubinger and colleagues trained models with backdoored behaviour and reported that “the backdoor behavior is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training”, and that adversarial training in particular “can teach models to better recognize their backdoor triggers, effectively hiding” the behaviour rather than removing it [14]. That is a finding about the limits of post-hoc remediation: a checkpoint you did not train is not made trustworthy by fine-tuning it. Separately, Souly and colleagues ran what they describe as the largest pretraining poisoning experiments to date, across models from 600 million to 13 billion parameters trained on datasets from 6 billion to 260 billion tokens, and found that “250 poisoned documents similarly compromise models across all model and dataset sizes, despite the largest models training on more than 20 times more clean data” [13]. The practical reading is uncomfortable: the required poison is a near-constant count, not a percentage, so scale does not dilute the attack.
The weight files themselves. Distribution format is a separate exposure from training. Hugging Face’s own documentation is blunt that pickle, the default serialisation for PyTorch weights, permits code execution at load time: “There are dangerous arbitrary code execution attacks that can be perpetrated when you load a pickle file”, and the recommended mitigations are provenance-based — load from parties you trust, rely on signed commits, or use a format such as safetensors [16]. Their scanner, which enumerates imports without executing them, carries an explicit disclaimer that it “is not 100% foolproof” [16]. Loading a checkpoint is running a program.
Tool descriptions. A tool definition is a string that a third party writes and your application inserts into the model’s context so the model can decide when to call it. It is therefore instruction-bearing text from an external source, with the same status as a retrieved document, and with a privileged position near the top of the context. The MCP specification treats third-party servers as a first-class threat surface, cataloguing local server compromise, token passthrough and server-side request forgery through URLs the server supplies, and requiring explicit pre-configuration consent showing the exact command that will run, without truncation, before a client executes a locally configured server [6].
Retrieved documents. Zou and colleagues formalised knowledge-base corruption and reported an attack success rate of 90 percent when injecting five malicious texts per target question into a knowledge database containing millions of texts, adding that the defences they tested were insufficient [15]. Retrieval is not a read-only operation from a security standpoint; it is an ingestion path with an authority level you have implicitly granted.
Every deployed mitigation is probabilistic
Real mitigations exist and they help. It matters to be exact about what kind of help they are.
Model-level prioritisation. Wallace and colleagues proposed an instruction hierarchy that explicitly defines model behaviour when instructions of different privilege conflict, and reported that training for it “drastically increases robustness — even for attack types not seen during training — while imposing minimal degradations on standard capabilities” [10]. This is a substantial contribution and it is also, structurally, a learned prior over token sequences. It changes which instruction usually wins. It does not make the low-privilege instruction unreachable.
Continuous adaptive evaluation. Google DeepMind’s report on defending Gemini describes an adversarial evaluation framework that “deploys a suite of adaptive attack techniques to run continuously against past, current, and future versions of Gemini”, with findings fed back into model hardening [11]. This is the right practice, and its existence is the clearest available evidence that the problem is treated internally as ongoing rather than closed.
Layered system defences. Google’s public security blog describes measures “designed for each stage of the prompt lifecycle”, from model hardening through purpose-built classifiers to system-level safeguards, and states that the approach “compels adversaries to resort to methods that are either more easily identified or demand greater resources” [12]. That is a vendor assertion and should be read as one — but note what it carefully does not say. It claims cost imposition, not elimination. Taken at face value, it is an accurate description of what a probabilistic stack achieves.
Here is the modelling error to avoid. Suppose a request passes
and conclude that four layers at ten percent leakage each give one bypass in ten thousand. That number is an artefact of the independence assumption, and the assumption is the weakest part of the model. The layers are typically built from the same model family, trained on overlapping data, and sensitive to the same features of an input; their failures are positively correlated, so the true joint failure probability is larger than the product. Worse, an adaptive adversary is not sampling inputs at random. They are searching for an input in the region where the layers fail together — which is exactly the correlated tail the product form assumes away. The honest quantity is the joint probability under an adversarially chosen input distribution, and no one currently knows how to measure it in a way that transfers to a new attack.
What this means for risk acceptance. A control with an unknown, non-stationary and adversarially selected failure rate cannot be the sole barrier in front of an irreversible action. It can legitimately reduce expected frequency, which is worth real money in aggregate. It cannot bound worst-case impact, and risk acceptance for these systems should therefore be written against impact rather than frequency: not “we assess the residual likelihood at low” but “if this control fails completely on a single request, the maximum damage is bounded by the following, and we accept that.” If you cannot complete that second sentence, the architecture is not finished.
Where practitioners genuinely disagree. There are two positions in the current literature and they are not simply opposed. One holds that layered, model-level and system-level probabilistic defence raises attacker cost enough to make broadly capable agents deployable today, and points to continuous adaptive evaluation as the discipline that keeps it honest [11, 12]. The other holds that no probabilistic component may be load-bearing, and that security must come from system-level constructions that constrain what untrusted data can influence, accepting a measurable utility cost [8, 9]. The disagreement is not about whether models can be made perfectly robust — nobody argues that — but about what may carry weight in a security argument. Notably, overlapping groups of authors appear on both sides, which suggests a live methodological debate rather than two camps talking past each other.
The narrow gate
The containment architecture that actually works has four parts. None of them involves getting better at recognising malicious text.
One: least-privilege credentials, scoped per task rather than per system. The agent should hold the smallest authority that completes the job in front of it, issued for that job, and expiring with it. In practice this means separate read and write identities, no shared omnibus service account, no wildcard scopes, and elevation requested at the moment a privileged operation is attempted rather than granted at session start [6, 4]. The test is simple: enumerate what the credential can reach and read the list as a description of the worst possible outcome, because that is what it is.
Two: deterministic policy on irreversible actions. The decision to perform an act with external, unrecoverable consequences must not be made by the model. It should be made by a policy engine outside the model, evaluating the structured action and its arguments — recipient, amount, destination, scope — against rules that do not read the model’s natural-language justification, because that justification is downstream of the attacker’s text. CaMeL is the clearest published demonstration of this shape: it extracts control and data flows from the trusted query so that untrusted retrieved data “can never impact the program flow”, and enforces capability-based policies at the point tools are called [8]. The reported cost is the most useful number in this article: CaMeL solved 77 percent of AgentDojo tasks with provable security, against 84 percent for an undefended system [8]. That is roughly a seven-point utility tax for a security property that does not degrade under a smarter attack. Beurer-Kellner and colleagues generalise the approach into a set of design patterns with explicitly stated utility-versus-security trade-offs [9].
Three: engineered reversibility. Prefer staged over committed. A draft rather than a sent message, a proposed change rather than a merged one, a queued transfer with a hold window rather than a settled one, soft deletion with retention rather than destruction. Reversibility converts a class of security failures into operational incidents, and operational incidents are survivable. This is the cheapest of the four controls and the most often skipped, because it costs product latency rather than engineering time.
Four: audit at the level of the action. The logged unit must be the individual tool call — the action, its full arguments, the credential used, and the provenance of the content that was in context when it was decided — not the conversation. Conversation-level logs cannot answer the question an investigation actually asks, which is which retrieved document was in the window when the transfer was proposed. The MCP specification makes the corresponding negative case concretely: token passthrough leaves downstream logs showing “requests that appear to come from a different source with a different identity”, making incident investigation and auditing harder [6]. Preserving identity and provenance through every hop is an audit requirement before it is an access-control one.
Scenario, offered as illustration and not as a report of any incident. An internal support assistant is given a single mailbox credential with read and send permission across a shared queue, plus a retrieval tool over the knowledge base and the ability to render replies with inline images. A ticket arrives from outside containing text addressed to the assistant. Under the architecture above, three of the four controls have to fail simultaneously for a breach: the credential would have to be broad enough to read other customers’ threads, the send action would have to execute without a deterministic recipient-allowlist check, and the rendering surface would have to permit an outbound fetch to an arbitrary host. Under the common architecture — one service account, model-decided sending, permissive rendering — one injected paragraph is sufficient. The difference between those two systems is not model quality. It is four decisions made at design time.
Predictions, with the assumptions and the disconfirming observations
These are forecasts, separated from the sourced analysis above. Horizon: 8 August 2028. Shared assumptions: transformer-based models with a single undifferentiated input sequence remain the dominant deployed architecture; agentic tool use continues to grow; no publicly disclosed cryptographic or architectural mechanism for authenticated instruction provenance inside the model input appears.
One. Prompt injection will not be solved at the model layer. Observable indicator: system cards and lab reports continue to publish non-zero attack success rates under adaptive evaluation. Disconfirmed if a major laboratory publishes an adaptive-attack evaluation reporting zero success across a comparable suite and an independent group reproduces it.
Two. Assurance practice shifts from “what did you filter” to “what could the credential do”. Observable indicator: audit and procurement frameworks begin requiring per-action authority inventories and blast-radius statements for deployed agents. Disconfirmed if mainstream guidance in 2028 still centres input filtering and detection as the primary control.
Three. The utility tax of provable containment narrows but does not reach zero. Observable indicator: successors to CaMeL report the gap against an undefended baseline on AgentDojo-class suites, and the gap shrinks from roughly seven points without closing. Disconfirmed if a defence with a stated security guarantee reports parity with an undefended agent under adaptive attack and the result replicates.
Four. Artefact provenance becomes a procurement requirement rather than a best practice. Observable indicator: signed weights, safetensors-only ingestion policies and supply-chain attestation appear as contractual terms. Disconfirmed if code-executing checkpoint formats remain the unremarked default in enterprise model procurement in 2028.
What to take away
The reason a sectioned lock is worth looking at is that it settles an argument by inspection. All the mass is in the case; all the security is in a gate a few thousandths of an inch wide. Add more brass and you have a heavier lock, not a better one.
A model-integrated system is built the same way. The model is housing — large, impressive, load-bearing for capability, and not a security boundary at any thickness, because its input has no channel on which to mark some tokens inert. The gate is the set of authorities the system can exercise and the policy that governs the irreversible ones. Defend that, keep it narrow, make it deterministic, log every passage through it, and make as much as possible on the far side undoable.
Then treat every probabilistic mitigation for what it honestly is: a way of making the attacker work harder to reach a gate that was going to hold anyway.
