Then in LinkedIn: Write article → click into the body → paste (Ctrl+V). Headings, links and images come with it. The title usually pastes as the first line — cut it into LinkedIn's title field. back to the article

How AI Security Defenses Actually Work

Sandboxes bound what an agent can touch, classifiers screen what a model reads and writes, automated search hunts jailbreaks at scale, AI tools hunt bugs in real code, and SOC copilots reorder an analyst's queue.

A small billet-aluminium microVM host on a bright lab bench with its access hatch caught mid-latch, a thin ribbon control lead running from it into a separate capability-policy module whose single port is lit, with a row of sealed hosts receding on a rack behind

Isolation is not one wall but two: a sealed execution boundary around what the agent runs, and a narrow, separately enforced gate around what it is allowed to ask for. — Image prompt and art direction by Brecht Corbeel; generation pending.

Abstract

This article gives a mechanism-level account of five classes of defensive AI security engineering — sandboxing and capability-based isolation that bounds what an agent's actions can reach, guardrail classifiers that screen model input and output as a separately trained system, automated red-teaming pipelines that replace one-off adversarial testing with gradient-based and language-model-guided search run at machine scale, AI-assisted vulnerability discovery tools that pair language models with fuzzers, debuggers and static analysis to find and patch real software bugs, and AI-driven defensive automation such as security-operations copilots that ground language-model output in an organization's own telemetry. Each mechanism is examined through its published architecture and documented results, and situated inside the lifecycle framework that NIST and the UK National Cyber Security Centre use to organize AI security guidance.

Five machines wearing one name

“AI security” reads as a single field from a distance and stops looking like one as soon as an engineer opens the box. Earlier in this series, this publication argued that prompt injection is architectural — that a language model has no channel on which to mark part of its input inert, so the defensible boundary has to move off the prompt entirely and onto the authority a system is willing to exercise on an instruction’s behalf. That argument is about where the boundary has to go. It says little about what actually sits at that boundary once an organization builds it, and the honest answer is that “AI security” as practiced today names at least five separate machines, built by different teams for different purposes, sharing almost no code and very little theory.

One machine isolates what a model-driven agent’s actions can physically reach, using virtualization and permission systems that predate large language models by decades and were simply pointed at a new kind of unreliable caller. A second reads everything a model reads and writes, as a distinct system trained specifically to classify content rather than converse. A third searches, automatically and at a scale no team of humans can match, for the inputs that break the first two. A fourth turns a language model into a tool for finding and patching vulnerabilities in ordinary software — not AI vulnerabilities, but the buffer overflows and use-after-frees that have existed since C compilers did. A fifth sits inside a security operations centre and changes which of a flood of alerts a human analyst sees first.

This article goes through each as an engineered system: what it is built from, how it processes its input, what its published results show, and where its documented limits are. None is a complete solution and none of the organizations that build them claims otherwise; the point of looking closely is that “the model is guarded” collapses a great deal of real, differently matured engineering into one vague reassurance, and the differences between the five machines are what determines whether a given deployment is actually protected against a given threat.

Two boundaries, not one: isolating what an agent can touch

The first machine has nothing to do with language models at all. It is the execution environment an agent’s actions run inside, and the mechanisms that build it were developed for a much older problem: how do you let code you do not fully trust run on hardware you do control, without letting it reach anything else on that hardware.

Two designs dominate. The first is hardware-backed virtualization, minimized. Amazon’s Firecracker is the clearest public example: it pairs the Linux kernel’s built-in virtualization support with a deliberately minimal virtual machine monitor, exposing only the handful of emulated devices a container or serverless function actually needs — network and block storage — and nothing else, which keeps the code that mediates the isolation boundary small enough to audit and small enough that a compromise inside the guest has very little surface to attack on the way out [1]. A launcher component, the “jailer,” strips the virtual machine monitor’s own privileges down before it ever starts a guest, so that even the mediating process is running with less authority than it could have. The guarantee this buys is the strongest kind available on commodity hardware: the boundary between what the agent’s code can see and what the host can see is enforced by the CPU’s own virtualization instructions, not by an operating system’s bookkeeping.

The second design keeps the speed and density of an ordinary container while trying to approach that guarantee in software. Google’s gVisor interposes an entire reimplemented kernel, called the Sentry, between a sandboxed process and the real Linux kernel underneath it: every system call the sandboxed code makes is intercepted and served by the Sentry’s own logic, written in a memory-safe language, rather than being passed to the host [2]. A companion process, the Gofer, handles the narrow set of filesystem operations that need to reach outside the sandbox, so even file access goes through a second checkpoint rather than a direct path. The trade this makes explicit: two kernels an attacker must break in sequence rather than one, at the cost of reimplementing — and potentially getting wrong — a large fraction of what an operating system does.

A capability-policy control module on a lab bench with a slim access token caught half-inserted into its one permitted slot while a second, rejected token lies capped beside it, a workstation screen behind showing a short policy rule being written

Figure 1. A capability policy does not ask whether a request looks safe; it checks whether this exact call was already on the list, which is why the check can be a deterministic procedure instead of a judgment call. — Image prompt and art direction by Brecht Corbeel; generation pending.

Neither design says anything about which actions an agent is allowed to take once it is safely inside its box. That is a separate problem, and it has produced a separate mechanism: capability-based permission control applied specifically to the tool calls a language-model agent issues. Progent is a recent published example. It sits between an agent and the tools it can invoke, checking every tool call against a policy written in a small domain-specific language before the call executes, “checked against” that policy “through a deterministic procedure, enforcing the principle of least privilege” [3]. Deterministic is the load-bearing word: the policy engine is not a language model making a judgment call about whether a request looks suspicious, but code evaluating whether this exact tool call, with these exact arguments, was already on an explicit allow-list — behaviour that can be tested and audited the way any other access-control code can be. Progent lets a language model draft an initial policy from the task description and lets an automated solver decide whether a refinement needs human approval, but constrains that flexibility in one direction only: “the agent’s effective action space can only shrink without approval” [3]. An LLM may help draft the fence. It is never allowed to widen it unsupervised.

Put the two layers together and the result is a stack, not a single wall: a hardware- or kernel-mediated boundary around the execution environment itself, and a separately enforced, deterministic policy around what that environment may do to the outside world. A failure in one does not automatically defeat the other — precisely the property a single monolithic “sandbox” cannot offer.

Guardrails: a second system that reads everything the first one does

The second machine is a pair of classifiers, and understanding it starts by rejecting the metaphor built into the word “filter.” A filter implies a fixed, mechanical screen. A guardrail classifier is itself a trained model, with its own architecture, its own training data, and its own failure modes, running alongside the model it is meant to protect rather than inside it.

The simplest published version is Meta’s Llama Guard: a language model fine-tuned specifically for classification rather than conversation, which takes either a user’s prompt or the primary model’s response and scores it against an explicit safety-risk taxonomy [4]. Two design choices in it recur across every guardrail system built since. First, it classifies both directions of the conversation — a request that looks benign can still be answered in a way that is not, and treating input and output as two separate classification problems catches more than scoring the exchange once. Second, because it is itself an instruction-tuned model rather than a fixed classifier head, its taxonomy of what counts as a violation can be changed by changing its instructions, without retraining — which is also exactly what makes it a target: it inherits the same instruction-following surface that makes the primary model persuadable.

A twin-slot card cage on a bright lab bench holding two small classifier accelerator boards, one already seated and one caught mid-insertion, with paired workstation screens behind showing an input score and an output score each partway formed

Figure 2. An input classifier and an output classifier are two separate readers of the same conversation, trained on different distributions and failing in different places, which is why a guardrail is a pair of systems rather than one filter. — Image prompt and art direction by Brecht Corbeel; generation pending.

Anthropic’s Constitutional Classifiers push the same two-classifier architecture toward a specific, harder threat: universal jailbreaks, meaning a single adversarial technique engineered to work across a wide range of unrelated harmful requests rather than one crafted for a single prompt. The mechanism that trains the classifiers is what gives the system its name — instead of hand-labelling millions of examples of harmful and harmless text, the researchers generate synthetic training data from a written “constitution,” a set of natural-language rules stating what content is and is not permitted, and use that constitution to produce the labelled examples the classifiers are then trained on [5]. The published results are unusually concrete for this literature. Compared with an unprotected model, the classifiers reduced the success rate of jailbreak attempts from 86 percent to 4.4 percent. That reduction was not free: the system added roughly 23.7 percent to inference compute, and increased the rate at which the model refused entirely benign production traffic by 0.38 percentage points [5]. A later evaluation round, comprising more than three thousand hours of red-teaming against the system, found no universal jailbreak that could reliably elicit detailed harmful output across the full set of target queries the evaluators tested [5]. Read as an engineering result rather than a marketing claim, this is what a genuinely hardened classifier layer looks like: a large, quantified reduction in one failure mode, a quantified and nonzero cost in both compute and false refusals, and a resistance claim that is bounded by how long and how hard the system was actually attacked — not a claim of impossibility.

It is worth being explicit about what a guardrail classifier is not. It does not touch the primary model’s weights and does not change what that model is fundamentally capable of producing if the classifier layer is bypassed; because it is trained rather than proved, it carries the same probabilistic character this series has already argued is unavoidable for anything built on top of a model rather than around its authority. The genuine engineering advance is not that these systems eliminate that character. It is that they make the classifier a separately optimizable, separately measurable component, so its failure rate can be driven down and reported on its own terms instead of being folded invisibly into “the model’s” overall behaviour.

Automated red-teaming: search replaces the annual penetration test

The third machine exists because neither a sandbox’s permission list nor a classifier’s decision boundary can be verified by reading its specification. Both have to be tested against inputs designed to break them, and the central engineering development of the last four years has been to stop writing those test inputs by hand.

The earliest systematic version of this idea used one language model to attack another. Researchers at DeepMind generated adversarial test cases with a language model rather than a human red-teamer, ran them against a 280-billion-parameter target chatbot, and scored the results with an automated classifier for offensive content — surfacing tens of thousands of harmful replies across categories the paper lists explicitly: “offensive discussions of groups, fabricated contact information, private training data leakage, and multi-turn conversation harms” [6]. The paper’s real contribution is not any single discovered failure but the demonstration that the generation strategy itself is a tunable design choice, ranging from simply prompting a generator model up through reinforcement learning against the target’s own failure rate, each producing test cases of measurably different diversity and difficulty [6].

Two later techniques specialize the same idea into concrete, automated jailbreak-search algorithms, sitting at opposite ends of what the attacker is assumed to know. The Greedy Coordinate Gradient method, GCG, assumes access to the target model’s gradients — directly or through an open-weight surrogate — and combines greedy and gradient-based search to construct an adversarial suffix, a short string of tokens appended to an ordinary request and optimized purely to raise the probability of compliance; its authors report that suffixes built this way proved “quite transferable” to systems they never had gradient access to at all, including proprietary models reachable only through an API [7]. PAIR, by contrast, assumes nothing but black-box query access: a second language model plays the attacker, iteratively drafting a candidate jailbreak, observing how the target responds, and rewriting its next attempt accordingly, with no gradient information and no human in the loop, converging inside what its authors’ title states directly — twenty queries [8].

The shared structural fact underneath both techniques, and the reason they are described here as one machine rather than two, is what a search process buys once it is automated rather than performed by hand. A single human red-teamer trying prompt variations by hand might realistically test dozens of candidates against a target in a working day. An automated pipeline built on either GCG- or PAIR-style search can attempt thousands of candidates against a running target in the same interval, entirely unattended. If a given search strategy succeeds against a defended target with independent per-attempt probability p, the probability that at least one of k automated attempts succeeds is

P_{\mathrm{hit}}(k) = 1 - (1-p)^{k}.

Even a strategy with a very low per-attempt success rate becomes a near-certain eventual hit once k is pushed into the thousands, which a machine can do overnight and a human red-team cannot do at all. This is precisely why Anthropic’s red-teaming of its classifier system is measured in thousands of hours rather than a handful of manual sessions [5], and it is the direct engineering rationale for building automated red-teaming as a standing pipeline rather than an annual exercise: the defender has to be able to generate and evaluate a search budget that at least approaches what an automated attacker could bring, or the defensive evaluation is measuring the wrong regime entirely.

A short server rack in a bright machine room with one compute tray drawn part-way out mid-job, a patch panel above it carrying many parallel leads, one lead caught landing in a single collector port

Figure 3. An automated pipeline does not need a cleverer attack than a person could write; it needs to try an enormous number of mediocre ones and keep only the failures that land, which is a property of throughput rather than of insight. — Image prompt and art direction by Brecht Corbeel; generation pending.

Teaching software to find its own bugs

The fourth machine is easy to mistake for the third because both involve a language model searching for a failure. The target is different in a way that matters: automated red-teaming searches for inputs that make a language model misbehave. AI-assisted vulnerability discovery searches for defects in ordinary software — memory-safety bugs, logic errors, injection flaws — using a language model as one component of the search, and it has produced working results against real, deployed code.

The most mature version augments a technique that already existed: coverage-guided fuzzing, in which a fuzzer feeds a program semi-random inputs and uses code-coverage feedback to steer future inputs toward unexplored paths. The bottleneck in fuzzing an unfamiliar library has never been the fuzzer itself; it is the fuzz harness, the small piece of code translating the fuzzer’s raw byte stream into calls against the library’s actual API, which an engineer traditionally has to write by hand for every new target. Google’s OSS-Fuzz project began using large language models specifically to write that harness code, later extending the same models to repair the compilation errors the generated harnesses produced and to triage the crashes the resulting runs turned up [9]. The model’s job is narrow — write the adapter, not search for the bug — while the search itself still runs on the same coverage-guided fuzzing engines that predate any of this by a decade. The results are concrete: dozens of genuine vulnerabilities surfaced in open-source libraries, including flaws that had gone undetected for years [9].

A lab bench with a small device-under-test board wired to a probe clip caught mid-contact, and a shallow sorting tray catching one flagged board mid-drop among a run of cleared ones

Figure 4. Most of what a fuzzing run produces is noise; the mechanism that matters is the one that decides, cheaply and often wrongly, which few results are worth a person's attention. — Image prompt and art direction by Brecht Corbeel; generation pending.

A second, structurally different approach gives the language model the reasoning role rather than the harness-writing role. Google’s Big Sleep agent, a collaboration between Project Zero and Google DeepMind, pairs a language model with a set of tools — a code browser, a debugger, a sandboxed execution environment, and shell access — and lets it work the way a human researcher does: reading a code change or commit message, forming a hypothesis about where a bug might have been introduced, constructing a test case, running it, and revising the hypothesis based on the result [10]. This is closer to automated variant analysis than to fuzzing; it reasons step by step about a specific piece of code rather than throwing large volumes of random input at a target. Its documented result is a stack buffer underflow in SQLite’s query-planning logic, found by exactly this hypothesize-and-test loop after the agent worked through the constraint-handling code around a specific class of column [10].

The most complete public demonstration of the fourth machine combined both approaches at competition scale. DARPA’s AI Cyber Challenge culminated in a finals event where seven teams’ automated “cyber reasoning systems” — narrowed from forty-two entrants — analyzed fifty-four million lines of real open-source code across sixty-three deliberately inserted vulnerabilities, while also free to find whatever genuine, previously unknown bugs existed in it [11]. The finalists found fifty-four of the sixty-three inserted vulnerabilities and patched forty-three of those; separately, they discovered eighteen real, previously unknown vulnerabilities and produced working patches for eleven, all responsibly disclosed to maintainers, at an average cost of roughly one hundred fifty-two dollars per task and an average patch turnaround of forty-five minutes [11]. Team Atlanta (Georgia Tech, KAIST, POSTECH and Samsung Research) won first place and four million dollars; Trail of Bits placed second with three million; Theori placed third with one and a half million [11, 12]. Trail of Bits’ own description of their system, Buttercup, names the fourth machine’s architecture about as concretely as any public source does: “a multi-agent architecture for intelligent patching with separation of concerns,” which “augments fuzzing tools (libFuzzer and Jazzer) with LLM-generated test cases” while separately integrating “static analysis tools like tree-sitter and code query systems” to reason about call graphs and vulnerability context before proposing a fix [12]. That sentence maps the fourth machine’s design space almost completely: fuzzing augmented by model-generated inputs, static analysis constraining where the model’s reasoning is applied, and both feeding one patching pipeline rather than separate tools a human has to reconcile by hand.

The SOC learns to read faster than it can hire

The fifth machine does not find vulnerabilities or block jailbreaks. It processes the output of everything else a security team already runs — endpoint alerts, identity logs, threat-intelligence feeds, the very tools described above — and its mechanism is worth being precise about because “AI in the SOC” is frequently described as if the model itself were doing detection, which is not how the deployed systems actually work.

Microsoft’s own architecture description of Security Copilot lays out the pipeline directly. A user’s prompt, or one triggered automatically from a connected product such as Defender or Sentinel, first passes through what Microsoft calls a grounding step: before the language model sees the request, the system checks it against the user’s identity and permissions, then uses plugins connected to an organization’s own security products and curated threat-intelligence sources to retrieve the specific, current context the request needs [13]. Only that grounded, enriched prompt reaches the model, whose response is then post-processed through the same plugin layer, which is how the system attaches citations and links a suggested action back to the specific alert that motivated it rather than simply asserting a conclusion [13]. The model here is a reasoning layer sitting between two rounds of retrieval and permission-checking, not a free-standing detector reading raw telemetry on its own authority — which is why the identity check happens before retrieval: the model must never be handed data the requesting analyst was not already entitled to see.

A security-operations desk with a physical console of small alert-token modules arranged in a queue, one token lifted just clear of its slot toward a priority rail, a headset resting on the desk and a workstation screen behind showing a short case summary

Figure 5. What a copilot changes first is not detection but ordering; the same alerts arrive, and the visible difference is which one is lifted out of the queue and placed in front of a person first. — Image prompt and art direction by Brecht Corbeel; generation pending.

What this architecture changes, in practice, is triage ordering rather than detection coverage. The underlying detection rules, threat-intelligence matches and anomaly scores mostly come from the same instrumented products a SOC already ran before any copilot was added; a grounded language-model layer adds the capacity to synthesize several signals into one scored, explained case faster than an analyst reading each source separately could, which changes which alert a human looks at first far more than it changes which alerts exist. That distinction matters for evaluating vendor claims: an assertion that a copilot “helps analysts work at machine speed and scale,” as Microsoft’s own documentation phrases it, is a claim about throughput and ordering, not about detecting threats the underlying products would otherwise have missed [13].

One more piece of infrastructure underwrites both this machine and the third: a shared vocabulary for what an attack against an AI system consists of. MITRE ATLAS catalogs adversary tactics and techniques against AI systems the way MITRE’s older ATT&CK framework catalogs them for conventional IT, giving a red-teaming pipeline, a SOC copilot’s plugin layer, and a governance team writing a risk register the same named techniques to test against and report coverage of, rather than each building a private taxonomy no one else can compare against [14]. A shared taxonomy is not itself a defensive mechanism, but it is the connective tissue that lets the mechanisms above be audited as a system rather than assessed one product brochure at a time.

The lifecycle these five mechanisms actually live inside

None of the five machines above is deployed in isolation, and the joint guidance issued by the UK’s National Cyber Security Centre together with the US Cybersecurity and Infrastructure Security Agency and a wide set of partner agencies gives the clearest public framework for where each one belongs. The guidelines organize AI system security into four lifecycle phases — secure design, secure development, secure deployment, and secure operation and maintenance — and state plainly that security has to be a requirement “throughout the life cycle of the system,” not a property checked once before launch [15].

Mapped against that framework, the five machines cluster cleanly. Isolation architecture and capability-based tool mediation are deployment-phase controls, worthless if designed only after deployment has already begun. Guardrail classifiers straddle deployment and operation, running continuously against live traffic but developed and versioned like any other model. Automated red-teaming and AI-assisted vulnerability discovery both belong squarely in the secure-development phase — NIST’s companion guidance to the same framework, developed jointly with CISA under Executive Order 14110, extends the government’s existing Secure Software Development Framework with practices specific to generative AI and dual-use foundation models, aimed at model producers, the developers who build systems around them, and the organizations that acquire those systems [16]. SOC automation is unambiguously an operation-and-maintenance control: it exists only because a system is already running and generating telemetry that has to be watched.

This mapping matters because it explains a failure pattern this series will return to elsewhere: an organization can deploy a genuinely well-engineered guardrail classifier and a genuinely well-designed sandbox and still be exposed, because neither control does anything for the development-phase question of whether the underlying software has exploitable bugs, or the operations-phase question of whether anyone is watching the telemetry once the system is live. The five machines are not redundant with each other. Each closes a gap at a different point in the lifecycle, and a posture built from only one or two of them is missing entire phases by construction, not by oversight.

Predictions, with the assumptions and the disconfirming observations

These are forecasts, separated from the sourced analysis above. Horizon: 16 August 2028. Shared assumptions: agentic systems with tool access continue to be deployed at increasing scale; no architectural change eliminates the underlying need for external mediation of model actions; automated search remains cheaper per attempt than human red-teaming by orders of magnitude.

One. Capability-based, deterministic policy engines for agent tool calls become a standard deployment component for production agents handling consequential actions, rather than a research prototype. Observable indicator: major agent frameworks ship a built-in policy-enforcement layer, not merely documentation recommending one. Disconfirmed if production agents in 2028 still rely primarily on the model’s own judgment to decide whether a tool call is appropriate.

Two. Guardrail classifier evaluations increasingly disclose both the attack-success reduction and the false-refusal cost together, following the pattern Constitutional Classifiers set, rather than either number alone. Observable indicator: major lab safety reports pair a jailbreak-resistance figure with a stated over-refusal rate. Disconfirmed if 2028 system cards still report robustness improvements without a paired cost figure.

Three. AI-assisted vulnerability discovery shifts from a research demonstration to a mandatory step in the software supply chain for critical infrastructure, driven by results like AIxCC’s rather than by vendor marketing. Observable indicator: procurement standards for critical-infrastructure software begin requiring documented automated vulnerability-discovery coverage. Disconfirmed if by 2028 such requirements remain confined to voluntary competitions and pilots.

Four. SOC copilots’ value proposition is measured chiefly in analyst throughput and mean time to triage rather than in detections attributed to the copilot, with vendors held to that framing by procurement contracts rather than their own restraint. Observable indicator: procurement benchmarks standardize on triage-latency metrics. Disconfirmed if 2028 marketing still centres unverifiable claims of threats “caught” by the copilot rather than measured effects on analyst workflow.

What to take away

Five machines, five different jobs. Isolation architecture and capability-based permission control decide what an agent’s actions can physically and logically reach. Guardrail classifiers are a second, separately trained system reading the same conversation from outside, with a measurable and nonzero cost. Automated red-teaming replaces the annual penetration test with continuous, machine-scale search, because a budget of thousands of automated attempts finds what a handful of manual ones cannot. AI-assisted vulnerability discovery turns a language model loose on ordinary software bugs, ranging from harness-writing for existing fuzzers to full reasoning agents with their own debuggers, with results now documented against real, previously unknown vulnerabilities in production code. Defensive automation in the SOC does not detect on its own authority; it grounds a language model in an organization’s own permissioned telemetry and changes which alert a person sees first.

None of the five closes the gap the rest of this series has spent its time on — that a model has no channel to mark part of its input inert, and every probabilistic layer built on top of that fact remains probabilistic no matter how well engineered. What the five machines do instead is make specific, separately measurable, separately testable pieces of a larger defensive architecture, each occupying its own phase of the system’s lifecycle. Ask which machine a given claim actually describes, at what lifecycle phase it operates, and what number backs it, and most of the vague reassurance in this field collapses into something you can evaluate.

Sources

  1. Firecracker Project Contributors. Firecracker. Amazon Web Services (2026).
  2. gVisor Project Contributors. Introduction to gVisor Security. gVisor Project (Google) (2026).
  3. Tianneng Shi, Jingxuan He, Zhun Wang, Hongwei Li, Linyu Wu, Wenbo Guo, and Dawn Song. Progent: Securing AI Agents with Privilege Control. arXiv (2025). DOI: 10.48550/arXiv.2504.11703.
  4. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv (2023). DOI: 10.48550/arXiv.2312.06674.
  5. Mrinank Sharma et al.. Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming. arXiv (2025). DOI: 10.48550/arXiv.2501.18837.
  6. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red Teaming Language Models with Language Models. Conference on Empirical Methods in Natural Language Processing (2022). DOI: 10.48550/arXiv.2202.03286.
  7. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv (2023). DOI: 10.48550/arXiv.2307.15043.
  8. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. Jailbreaking Black Box Large Language Models in Twenty Queries. arXiv (2023). DOI: 10.48550/arXiv.2310.08419.
  9. Google Open Source Security Team. AI-Powered Fuzzing: Breaking the Bug Hunting Barrier. Google Online Security Blog (2023).
  10. Google Project Zero and Google DeepMind. From Naptime to Big Sleep: Using Large Language Models To Catch Vulnerabilities In Real-World Code. Project Zero (2024).
  11. Defense Advanced Research Projects Agency. AI Cyber Challenge Marks Pivotal Inflection Point for Cyber Defense. DARPA (2025).
  12. Trail of Bits. Trail of Bits' Buttercup Wins 2nd Place in AIxCC Challenge. Trail of Bits Blog (2025).
  13. Microsoft. What Is Microsoft Security Copilot?. Microsoft Learn (2026).
  14. MITRE Corporation. MITRE ATLAS. MITRE (2026).
  15. UK National Cyber Security Centre, US Cybersecurity and Infrastructure Security Agency, and partner agencies. Guidelines for Secure AI System Development. National Cyber Security Centre (2023).
  16. Harold Booth, Murugiah Souppaya, Apostol Vassilev, Michael Ogata, Martin Stanley, and Karen Scarfone. Secure Software Development Practices for Generative AI and Dual-Use Foundation Models: An SSDF Community Profile. National Institute of Standards and Technology (2024).

Originally published at https://absolutedigitalpublishers.com/articles/how-ai-security-defenses-actually-work.