Five machines wearing one name
“AI security” reads as a single field from a distance and stops looking like one as soon as an engineer opens the box. Earlier in this series, this publication argued that prompt injection is architectural — that a language model has no channel on which to mark part of its input inert, so the defensible boundary has to move off the prompt entirely and onto the authority a system is willing to exercise on an instruction’s behalf. That argument is about where the boundary has to go. It says little about what actually sits at that boundary once an organization builds it, and the honest answer is that “AI security” as practiced today names at least five separate machines, built by different teams for different purposes, sharing almost no code and very little theory.
One machine isolates what a model-driven agent’s actions can physically reach, using virtualization and permission systems that predate large language models by decades and were simply pointed at a new kind of unreliable caller. A second reads everything a model reads and writes, as a distinct system trained specifically to classify content rather than converse. A third searches, automatically and at a scale no team of humans can match, for the inputs that break the first two. A fourth turns a language model into a tool for finding and patching vulnerabilities in ordinary software — not AI vulnerabilities, but the buffer overflows and use-after-frees that have existed since C compilers did. A fifth sits inside a security operations centre and changes which of a flood of alerts a human analyst sees first.
This article goes through each as an engineered system: what it is built from, how it processes its input, what its published results show, and where its documented limits are. None is a complete solution and none of the organizations that build them claims otherwise; the point of looking closely is that “the model is guarded” collapses a great deal of real, differently matured engineering into one vague reassurance, and the differences between the five machines are what determines whether a given deployment is actually protected against a given threat.
Two boundaries, not one: isolating what an agent can touch
The first machine has nothing to do with language models at all. It is the execution environment an agent’s actions run inside, and the mechanisms that build it were developed for a much older problem: how do you let code you do not fully trust run on hardware you do control, without letting it reach anything else on that hardware.
Two designs dominate. The first is hardware-backed virtualization, minimized. Amazon’s Firecracker is the clearest public example: it pairs the Linux kernel’s built-in virtualization support with a deliberately minimal virtual machine monitor, exposing only the handful of emulated devices a container or serverless function actually needs — network and block storage — and nothing else, which keeps the code that mediates the isolation boundary small enough to audit and small enough that a compromise inside the guest has very little surface to attack on the way out [1]. A launcher component, the “jailer,” strips the virtual machine monitor’s own privileges down before it ever starts a guest, so that even the mediating process is running with less authority than it could have. The guarantee this buys is the strongest kind available on commodity hardware: the boundary between what the agent’s code can see and what the host can see is enforced by the CPU’s own virtualization instructions, not by an operating system’s bookkeeping.
The second design keeps the speed and density of an ordinary container while trying to approach that guarantee in software. Google’s gVisor interposes an entire reimplemented kernel, called the Sentry, between a sandboxed process and the real Linux kernel underneath it: every system call the sandboxed code makes is intercepted and served by the Sentry’s own logic, written in a memory-safe language, rather than being passed to the host [2]. A companion process, the Gofer, handles the narrow set of filesystem operations that need to reach outside the sandbox, so even file access goes through a second checkpoint rather than a direct path. The trade this makes explicit: two kernels an attacker must break in sequence rather than one, at the cost of reimplementing — and potentially getting wrong — a large fraction of what an operating system does.
Neither design says anything about which actions an agent is allowed to take once it is safely inside its box. That is a separate problem, and it has produced a separate mechanism: capability-based permission control applied specifically to the tool calls a language-model agent issues. Progent is a recent published example. It sits between an agent and the tools it can invoke, checking every tool call against a policy written in a small domain-specific language before the call executes, “checked against” that policy “through a deterministic procedure, enforcing the principle of least privilege” [3]. Deterministic is the load-bearing word: the policy engine is not a language model making a judgment call about whether a request looks suspicious, but code evaluating whether this exact tool call, with these exact arguments, was already on an explicit allow-list — behaviour that can be tested and audited the way any other access-control code can be. Progent lets a language model draft an initial policy from the task description and lets an automated solver decide whether a refinement needs human approval, but constrains that flexibility in one direction only: “the agent’s effective action space can only shrink without approval” [3]. An LLM may help draft the fence. It is never allowed to widen it unsupervised.
Put the two layers together and the result is a stack, not a single wall: a hardware- or kernel-mediated boundary around the execution environment itself, and a separately enforced, deterministic policy around what that environment may do to the outside world. A failure in one does not automatically defeat the other — precisely the property a single monolithic “sandbox” cannot offer.
Guardrails: a second system that reads everything the first one does
The second machine is a pair of classifiers, and understanding it starts by rejecting the metaphor built into the word “filter.” A filter implies a fixed, mechanical screen. A guardrail classifier is itself a trained model, with its own architecture, its own training data, and its own failure modes, running alongside the model it is meant to protect rather than inside it.
The simplest published version is Meta’s Llama Guard: a language model fine-tuned specifically for classification rather than conversation, which takes either a user’s prompt or the primary model’s response and scores it against an explicit safety-risk taxonomy [4]. Two design choices in it recur across every guardrail system built since. First, it classifies both directions of the conversation — a request that looks benign can still be answered in a way that is not, and treating input and output as two separate classification problems catches more than scoring the exchange once. Second, because it is itself an instruction-tuned model rather than a fixed classifier head, its taxonomy of what counts as a violation can be changed by changing its instructions, without retraining — which is also exactly what makes it a target: it inherits the same instruction-following surface that makes the primary model persuadable.
Anthropic’s Constitutional Classifiers push the same two-classifier architecture toward a specific, harder threat: universal jailbreaks, meaning a single adversarial technique engineered to work across a wide range of unrelated harmful requests rather than one crafted for a single prompt. The mechanism that trains the classifiers is what gives the system its name — instead of hand-labelling millions of examples of harmful and harmless text, the researchers generate synthetic training data from a written “constitution,” a set of natural-language rules stating what content is and is not permitted, and use that constitution to produce the labelled examples the classifiers are then trained on [5]. The published results are unusually concrete for this literature. Compared with an unprotected model, the classifiers reduced the success rate of jailbreak attempts from 86 percent to 4.4 percent. That reduction was not free: the system added roughly 23.7 percent to inference compute, and increased the rate at which the model refused entirely benign production traffic by 0.38 percentage points [5]. A later evaluation round, comprising more than three thousand hours of red-teaming against the system, found no universal jailbreak that could reliably elicit detailed harmful output across the full set of target queries the evaluators tested [5]. Read as an engineering result rather than a marketing claim, this is what a genuinely hardened classifier layer looks like: a large, quantified reduction in one failure mode, a quantified and nonzero cost in both compute and false refusals, and a resistance claim that is bounded by how long and how hard the system was actually attacked — not a claim of impossibility.
It is worth being explicit about what a guardrail classifier is not. It does not touch the primary model’s weights and does not change what that model is fundamentally capable of producing if the classifier layer is bypassed; because it is trained rather than proved, it carries the same probabilistic character this series has already argued is unavoidable for anything built on top of a model rather than around its authority. The genuine engineering advance is not that these systems eliminate that character. It is that they make the classifier a separately optimizable, separately measurable component, so its failure rate can be driven down and reported on its own terms instead of being folded invisibly into “the model’s” overall behaviour.
Automated red-teaming: search replaces the annual penetration test
The third machine exists because neither a sandbox’s permission list nor a classifier’s decision boundary can be verified by reading its specification. Both have to be tested against inputs designed to break them, and the central engineering development of the last four years has been to stop writing those test inputs by hand.
The earliest systematic version of this idea used one language model to attack another. Researchers at DeepMind generated adversarial test cases with a language model rather than a human red-teamer, ran them against a 280-billion-parameter target chatbot, and scored the results with an automated classifier for offensive content — surfacing tens of thousands of harmful replies across categories the paper lists explicitly: “offensive discussions of groups, fabricated contact information, private training data leakage, and multi-turn conversation harms” [6]. The paper’s real contribution is not any single discovered failure but the demonstration that the generation strategy itself is a tunable design choice, ranging from simply prompting a generator model up through reinforcement learning against the target’s own failure rate, each producing test cases of measurably different diversity and difficulty [6].
Two later techniques specialize the same idea into concrete, automated jailbreak-search algorithms, sitting at opposite ends of what the attacker is assumed to know. The Greedy Coordinate Gradient method, GCG, assumes access to the target model’s gradients — directly or through an open-weight surrogate — and combines greedy and gradient-based search to construct an adversarial suffix, a short string of tokens appended to an ordinary request and optimized purely to raise the probability of compliance; its authors report that suffixes built this way proved “quite transferable” to systems they never had gradient access to at all, including proprietary models reachable only through an API [7]. PAIR, by contrast, assumes nothing but black-box query access: a second language model plays the attacker, iteratively drafting a candidate jailbreak, observing how the target responds, and rewriting its next attempt accordingly, with no gradient information and no human in the loop, converging inside what its authors’ title states directly — twenty queries [8].
The shared structural fact underneath both techniques, and the reason they are described here as one machine rather than two, is what a search process buys once it is automated rather than performed by hand. A single human red-teamer trying prompt variations by hand might realistically test dozens of candidates against a target in a working day. An automated pipeline built on either GCG- or PAIR-style search can attempt thousands of candidates against a running target in the same interval, entirely unattended. If a given search strategy succeeds against a defended target with independent per-attempt probability
Even a strategy with a very low per-attempt success rate becomes a near-certain eventual hit once
Teaching software to find its own bugs
The fourth machine is easy to mistake for the third because both involve a language model searching for a failure. The target is different in a way that matters: automated red-teaming searches for inputs that make a language model misbehave. AI-assisted vulnerability discovery searches for defects in ordinary software — memory-safety bugs, logic errors, injection flaws — using a language model as one component of the search, and it has produced working results against real, deployed code.
The most mature version augments a technique that already existed: coverage-guided fuzzing, in which a fuzzer feeds a program semi-random inputs and uses code-coverage feedback to steer future inputs toward unexplored paths. The bottleneck in fuzzing an unfamiliar library has never been the fuzzer itself; it is the fuzz harness, the small piece of code translating the fuzzer’s raw byte stream into calls against the library’s actual API, which an engineer traditionally has to write by hand for every new target. Google’s OSS-Fuzz project began using large language models specifically to write that harness code, later extending the same models to repair the compilation errors the generated harnesses produced and to triage the crashes the resulting runs turned up [9]. The model’s job is narrow — write the adapter, not search for the bug — while the search itself still runs on the same coverage-guided fuzzing engines that predate any of this by a decade. The results are concrete: dozens of genuine vulnerabilities surfaced in open-source libraries, including flaws that had gone undetected for years [9].
A second, structurally different approach gives the language model the reasoning role rather than the harness-writing role. Google’s Big Sleep agent, a collaboration between Project Zero and Google DeepMind, pairs a language model with a set of tools — a code browser, a debugger, a sandboxed execution environment, and shell access — and lets it work the way a human researcher does: reading a code change or commit message, forming a hypothesis about where a bug might have been introduced, constructing a test case, running it, and revising the hypothesis based on the result [10]. This is closer to automated variant analysis than to fuzzing; it reasons step by step about a specific piece of code rather than throwing large volumes of random input at a target. Its documented result is a stack buffer underflow in SQLite’s query-planning logic, found by exactly this hypothesize-and-test loop after the agent worked through the constraint-handling code around a specific class of column [10].
The most complete public demonstration of the fourth machine combined both approaches at competition scale. DARPA’s AI Cyber Challenge culminated in a finals event where seven teams’ automated “cyber reasoning systems” — narrowed from forty-two entrants — analyzed fifty-four million lines of real open-source code across sixty-three deliberately inserted vulnerabilities, while also free to find whatever genuine, previously unknown bugs existed in it [11]. The finalists found fifty-four of the sixty-three inserted vulnerabilities and patched forty-three of those; separately, they discovered eighteen real, previously unknown vulnerabilities and produced working patches for eleven, all responsibly disclosed to maintainers, at an average cost of roughly one hundred fifty-two dollars per task and an average patch turnaround of forty-five minutes [11]. Team Atlanta (Georgia Tech, KAIST, POSTECH and Samsung Research) won first place and four million dollars; Trail of Bits placed second with three million; Theori placed third with one and a half million [11, 12]. Trail of Bits’ own description of their system, Buttercup, names the fourth machine’s architecture about as concretely as any public source does: “a multi-agent architecture for intelligent patching with separation of concerns,” which “augments fuzzing tools (libFuzzer and Jazzer) with LLM-generated test cases” while separately integrating “static analysis tools like tree-sitter and code query systems” to reason about call graphs and vulnerability context before proposing a fix [12]. That sentence maps the fourth machine’s design space almost completely: fuzzing augmented by model-generated inputs, static analysis constraining where the model’s reasoning is applied, and both feeding one patching pipeline rather than separate tools a human has to reconcile by hand.
The SOC learns to read faster than it can hire
The fifth machine does not find vulnerabilities or block jailbreaks. It processes the output of everything else a security team already runs — endpoint alerts, identity logs, threat-intelligence feeds, the very tools described above — and its mechanism is worth being precise about because “AI in the SOC” is frequently described as if the model itself were doing detection, which is not how the deployed systems actually work.
Microsoft’s own architecture description of Security Copilot lays out the pipeline directly. A user’s prompt, or one triggered automatically from a connected product such as Defender or Sentinel, first passes through what Microsoft calls a grounding step: before the language model sees the request, the system checks it against the user’s identity and permissions, then uses plugins connected to an organization’s own security products and curated threat-intelligence sources to retrieve the specific, current context the request needs [13]. Only that grounded, enriched prompt reaches the model, whose response is then post-processed through the same plugin layer, which is how the system attaches citations and links a suggested action back to the specific alert that motivated it rather than simply asserting a conclusion [13]. The model here is a reasoning layer sitting between two rounds of retrieval and permission-checking, not a free-standing detector reading raw telemetry on its own authority — which is why the identity check happens before retrieval: the model must never be handed data the requesting analyst was not already entitled to see.
What this architecture changes, in practice, is triage ordering rather than detection coverage. The underlying detection rules, threat-intelligence matches and anomaly scores mostly come from the same instrumented products a SOC already ran before any copilot was added; a grounded language-model layer adds the capacity to synthesize several signals into one scored, explained case faster than an analyst reading each source separately could, which changes which alert a human looks at first far more than it changes which alerts exist. That distinction matters for evaluating vendor claims: an assertion that a copilot “helps analysts work at machine speed and scale,” as Microsoft’s own documentation phrases it, is a claim about throughput and ordering, not about detecting threats the underlying products would otherwise have missed [13].
One more piece of infrastructure underwrites both this machine and the third: a shared vocabulary for what an attack against an AI system consists of. MITRE ATLAS catalogs adversary tactics and techniques against AI systems the way MITRE’s older ATT&CK framework catalogs them for conventional IT, giving a red-teaming pipeline, a SOC copilot’s plugin layer, and a governance team writing a risk register the same named techniques to test against and report coverage of, rather than each building a private taxonomy no one else can compare against [14]. A shared taxonomy is not itself a defensive mechanism, but it is the connective tissue that lets the mechanisms above be audited as a system rather than assessed one product brochure at a time.
The lifecycle these five mechanisms actually live inside
None of the five machines above is deployed in isolation, and the joint guidance issued by the UK’s National Cyber Security Centre together with the US Cybersecurity and Infrastructure Security Agency and a wide set of partner agencies gives the clearest public framework for where each one belongs. The guidelines organize AI system security into four lifecycle phases — secure design, secure development, secure deployment, and secure operation and maintenance — and state plainly that security has to be a requirement “throughout the life cycle of the system,” not a property checked once before launch [15].
Mapped against that framework, the five machines cluster cleanly. Isolation architecture and capability-based tool mediation are deployment-phase controls, worthless if designed only after deployment has already begun. Guardrail classifiers straddle deployment and operation, running continuously against live traffic but developed and versioned like any other model. Automated red-teaming and AI-assisted vulnerability discovery both belong squarely in the secure-development phase — NIST’s companion guidance to the same framework, developed jointly with CISA under Executive Order 14110, extends the government’s existing Secure Software Development Framework with practices specific to generative AI and dual-use foundation models, aimed at model producers, the developers who build systems around them, and the organizations that acquire those systems [16]. SOC automation is unambiguously an operation-and-maintenance control: it exists only because a system is already running and generating telemetry that has to be watched.
This mapping matters because it explains a failure pattern this series will return to elsewhere: an organization can deploy a genuinely well-engineered guardrail classifier and a genuinely well-designed sandbox and still be exposed, because neither control does anything for the development-phase question of whether the underlying software has exploitable bugs, or the operations-phase question of whether anyone is watching the telemetry once the system is live. The five machines are not redundant with each other. Each closes a gap at a different point in the lifecycle, and a posture built from only one or two of them is missing entire phases by construction, not by oversight.
Predictions, with the assumptions and the disconfirming observations
These are forecasts, separated from the sourced analysis above. Horizon: 16 August 2028. Shared assumptions: agentic systems with tool access continue to be deployed at increasing scale; no architectural change eliminates the underlying need for external mediation of model actions; automated search remains cheaper per attempt than human red-teaming by orders of magnitude.
One. Capability-based, deterministic policy engines for agent tool calls become a standard deployment component for production agents handling consequential actions, rather than a research prototype. Observable indicator: major agent frameworks ship a built-in policy-enforcement layer, not merely documentation recommending one. Disconfirmed if production agents in 2028 still rely primarily on the model’s own judgment to decide whether a tool call is appropriate.
Two. Guardrail classifier evaluations increasingly disclose both the attack-success reduction and the false-refusal cost together, following the pattern Constitutional Classifiers set, rather than either number alone. Observable indicator: major lab safety reports pair a jailbreak-resistance figure with a stated over-refusal rate. Disconfirmed if 2028 system cards still report robustness improvements without a paired cost figure.
Three. AI-assisted vulnerability discovery shifts from a research demonstration to a mandatory step in the software supply chain for critical infrastructure, driven by results like AIxCC’s rather than by vendor marketing. Observable indicator: procurement standards for critical-infrastructure software begin requiring documented automated vulnerability-discovery coverage. Disconfirmed if by 2028 such requirements remain confined to voluntary competitions and pilots.
Four. SOC copilots’ value proposition is measured chiefly in analyst throughput and mean time to triage rather than in detections attributed to the copilot, with vendors held to that framing by procurement contracts rather than their own restraint. Observable indicator: procurement benchmarks standardize on triage-latency metrics. Disconfirmed if 2028 marketing still centres unverifiable claims of threats “caught” by the copilot rather than measured effects on analyst workflow.
What to take away
Five machines, five different jobs. Isolation architecture and capability-based permission control decide what an agent’s actions can physically and logically reach. Guardrail classifiers are a second, separately trained system reading the same conversation from outside, with a measurable and nonzero cost. Automated red-teaming replaces the annual penetration test with continuous, machine-scale search, because a budget of thousands of automated attempts finds what a handful of manual ones cannot. AI-assisted vulnerability discovery turns a language model loose on ordinary software bugs, ranging from harness-writing for existing fuzzers to full reasoning agents with their own debuggers, with results now documented against real, previously unknown vulnerabilities in production code. Defensive automation in the SOC does not detect on its own authority; it grounds a language model in an organization’s own permissioned telemetry and changes which alert a person sees first.
None of the five closes the gap the rest of this series has spent its time on — that a model has no channel to mark part of its input inert, and every probabilistic layer built on top of that fact remains probabilistic no matter how well engineered. What the five machines do instead is make specific, separately measurable, separately testable pieces of a larger defensive architecture, each occupying its own phase of the system’s lifecycle. Ask which machine a given claim actually describes, at what lifecycle phase it operates, and what number backs it, and most of the vague reassurance in this field collapses into something you can evaluate.
