Four questions, one shared problem

Every tool discussed in this article does the same underlying thing: it gives a language model the ability to read files, write files, and run commands against a real codebase. Once that ability exists, the tool has to answer four design questions, and the answers are what actually distinguish one product from another — not the marketing language, not the model behind it, and not any single benchmark score.

Who authorizes an action before it runs, and how coarse or fine is that authorization? What persists once the session ends, and who wrote it? How does work get delegated beyond a single thread of execution — to another process, another machine, another vendor’s tool? And what gets checked before a change is allowed to count as finished? This article works through four broad approaches found in shipping products today against those four axes: Claude Code’s permission-gated agentic loop, IDE-embedded inline copilots, autonomous cloud background agents, and plan-then-execute systems. The last of these is less a separate product category than a mode some tools adopt, but it answers the four questions differently enough to treat on its own terms.

The comparison that follows is explicitly not a ranking. Where the evidence supports a benchmark comparison, it says so and states the caveat. Where it does not — which is most of the time — it says that instead, and explains why the numbers on offer cannot be made commensurable.

ADVERTISEMENT

The permission-gated loop: Claude Code

Claude Code’s default unit of work is a single tool call — one shell command, one file edit, one web fetch — and its default answer to “who authorizes this” is: a person, asked in the moment. The documentation describes a tiered system in which rules are evaluated in a fixed order, deny before ask before allow, so that a deny rule always wins even when a more specific allow rule also matches the same call [1]. Read-only actions such as file reads and searches proceed without asking, within the working directory; shell commands require approval except for a built-in set of read-only commands; and a user can grant a specific command permanently for a given repository so the same question is not asked twice. These rules live in settings.json files at three scopes — user-level, shared project-level committed to version control, and organization-level managed policy — which means a team can commit a shared set of allow and deny rules that every contributor inherits, while each developer still layers personal preferences on top [1].

What makes this a genuinely different architecture from “the model decides, then asks” is that the gate does not have to run through the model’s own judgment at all. Hooks are shell commands, HTTP calls, or even other model invocations that fire automatically at fixed points in the loop — before a tool call, after it succeeds, after it fails, at the start and end of a session, and more than a dozen other named events. A PreToolUse hook can inspect the pending call and return a decision — allow, deny, or escalate to the user — and a script that exits with a specific status code blocks the action outright, with the reason surfaced back to the model as why it was stopped [2]. This is a structurally different kind of check than a permission prompt: a permission prompt asks a person to evaluate the model’s proposed action, while a hook is a piece of deterministic code that runs regardless of what the model would prefer, and fires the same way whether the call came from the main session or from a subagent working on its behalf [2].

A terminal desk with a monitor showing a paused command prompt, and a desktop stamp unit lowering onto a kraft approval slip just before it lands
Figure 1. Claude Code asks before most actions by default; a hook can instead force a hard stop that runs whether or not the model agrees to wait.Image prompt and art direction by Brecht Corbeel; image generated to that direction.

The embedded-suggestion approach: inline copilots

IDE-embedded copilots — the ancestor form of this category, and still the most common way most developers encounter an AI coding tool day to day — answer the same four questions very differently, mostly because the unit of work is much smaller. An inline suggestion is not a shell command with side effects; it is a proposed span of text that the editor renders as faint “ghost text” ahead of the cursor, and the checkpoint is simply whether the developer’s next keystroke accepts it, edits it, or keeps typing past it. There is no separate approval dialogue because there is nothing separate to approve — acceptance and rejection are folded into the ordinary act of writing code, at a granularity of a few tokens rather than a whole file or a whole command.

This has a real consequence for every other axis. Because the interaction stays inside a single edit buffer with a human’s hands on the keyboard the entire time, these tools have historically needed no delegation model, no durable cross-session memory beyond the open files and recent edits, and no hooks, because there is no unattended interval for a hook to guard. What memory they do carry is usually a single static instructions file rather than a layered hierarchy — Claude Code’s own /init command, for instance, explicitly reads a competing tool’s convention, .github/copilot-instructions.md, when assembling a new project’s instructions, precisely because that is the shape the inline-copilot family already uses [5]. The trade this approach makes is legible: minimal interaction overhead and a tight, continuously human-supervised loop, in exchange for an approach that does not by itself scale to a task that requires touching many files, running a test suite, or working while its user is away from the keyboard. Most products in this space have since grown an additional, separate agent mode to cover exactly that gap — which is itself evidence that the four questions above are not answered once per product, but potentially once per mode within a single product.

A mechanical keyboard beside a docked laptop, one key still rising back to height on its spring, with an external monitor's faint suggestion text visible out of sharp focus behind it
Figure 2. An inline copilot keeps its checkpoint at the keystroke itself — a suggestion offered and accepted or discarded one line at a time, never a shell command or a whole plan.Image prompt and art direction by Brecht Corbeel; image generated to that direction.

The autonomous background agent

A third family answers all four questions at once by moving the entire loop off the developer’s machine and into a disposable, unattended environment. GitHub’s Copilot coding agent is assigned a task from an issue, a pull request comment, or a scheduled trigger, and then works inside “its own ephemeral development environment, powered by GitHub Actions,” exploring the code, making changes, and running tests before opening a pull request — described in GitHub’s own documentation as happening “without human intervention during execution” [6]. The permission model here is front-loaded rather than continuous: a person authorizes the task once, at assignment, and the next checkpoint is the pull request itself. GitHub’s documentation is specific about the resulting constraints — a single session can only modify the one repository specified at the start, can only work on one branch and open exactly one pull request, and is capped at a maximum execution time of 59 minutes; branch protection rules can block it entirely unless it is explicitly added as a bypass actor [6].

ADVERTISEMENT

OpenAI’s Codex occupies the same family with a different internal split. Tasks run inside an isolated cloud sandbox with a documented two-layer control scheme: a technical sandbox that enforces what is possible at the operating-system level — via sandbox-exec on macOS, bwrap with seccomp filters on Linux — and a separate approval policy governing when the agent must ask a person first [7]. Three modes sit on that spectrum: a read-only mode that requires approval for any modification, an “auto” default that can read, edit, and run commands inside the workspace but asks before touching anything outside it or reaching the network, and a full-access mode that removes the sandbox and approval checks entirely. Network access is disabled by default even in auto mode, writes are confined to the active workspace, and specific paths — .git, .agents, .codex — stay read-only regardless of mode [7].

Cognition’s Devin, announced in 2024 as what the company called “the first AI software engineer,” is built around the same unattended-sandbox shape but foregrounds a planning step ahead of execution: the company’s own description states that Devin “plans and execute[s] complex engineering tasks requiring thousands of decisions” using “the shell, code editor, and browser within a sandboxed compute environment,” reporting progress and accepting feedback along the way [8]. That announcement also reported a benchmark figure worth treating carefully: Devin “correctly resolves 13.86% of the issues end-to-end” on SWE-bench, a dataset the company describes as testing whether a model can resolve real GitHub issues, which it characterizes as “far exceeding the previous state-of-the-art of 1.96%” [8]. That 1.96% baseline is a real number from the paper that introduced SWE-bench, which evaluated models against 2,294 real issue-and-pull-request pairs drawn from twelve popular Python repositories and found that the best model tested at the time, Claude 2, solved fewer than one in fifty [9].

Both numbers are worth stating plainly and then setting aside as a comparison tool. They come from different years, different evaluation harnesses, and — critically — the 13.86% figure is Cognition’s own reported result on its own scaffold, not an independent reproduction, so it is a vendor claim about a specific configuration rather than a fact about Devin in general. Neither number can be extended to say anything about Claude Code, Copilot’s coding agent, or any tool not actually run against a matched, dated, independently reproduced version of the benchmark. This article does not construct a ranking from them, and any comparison a reader encounters elsewhere that does should be read with the same caution.

An unattended desk with an idling laptop, its charge light lit, and a small courier-style out-tray in which a folder is caught just settling into place with no one present
Figure 3. A background agent works off in its own sandboxed environment and reports back only at the end; the desk stays unattended for the whole of that interval, not only at its finish.Image prompt and art direction by Brecht Corbeel; image generated to that direction.

Plan-then-execute, and the trade-off made explicit

A fourth pattern — present as a mode inside otherwise interactive tools as often as it is a standalone product — inverts where the single largest checkpoint sits. Rather than approving many small actions as they occur, or approving one task at assignment and reviewing only the finished pull request, a plan-then-execute system produces a complete, legible plan first and asks for approval of the plan itself before any tool call runs against the real codebase. Devin’s own description of planning “complex engineering tasks requiring thousands of decisions” before execution is one instance of this pattern; Claude Code ships a comparable mode of its own, built around a dedicated planning agent that researches a codebase and proposes an approach before any file is touched, rather than folding planning invisibly into the first few tool calls [3, 8].

The genuine trade this design makes can be stated without hand-waving. Suppose a task requires nn discrete tool calls before it is complete, and let gg be the number of those calls that execute between one human checkpoint and the next — the granularity of approval. A tool that asks before every action sets g=1g=1; a tool that asks once for an entire plan, or once for an entire background task, sets gg close to nn. At a fixed cost cc for each checkpoint a person has to evaluate, total interaction cost scales with the number of checkpoints:

Interaction costngc \text{Interaction cost} \approx \frac{n}{g}\,c

Now suppose each individual tool call has some small, independent probability pp of being consequential and wrong in a way a reviewer would have caught. The probability that at least one such call executes, unreviewed, inside a single gap of gg calls is

ADVERTISEMENT
Exposure per gap=1(1p)g. \text{Exposure per gap} = 1-(1-p)^g.

This is the same “at least one” shape that shows up whenever independent trials are combined, applied here to the opposite question from the one it usually answers: not how many attempts are needed before one succeeds, but how many unwatched actions occur before one goes wrong. For small gg, exposure grows almost linearly — each additional ungated action adds roughly pp more exposure — so coarsening from asking every time to asking every fifth action multiplies exposure roughly fivefold while cutting interaction cost by the same factor: a real trade, not a free improvement in either direction. But the curve saturates: coarsening further, from a handful of calls to an entire multi-hour unattended task where gg runs into the hundreds, buys comparatively little additional reduction in interaction cost per call, because the checkpoint count is already near its floor of one, while exposure is already close to its ceiling. Read against that shape, Claude Code’s default of asking per action, a plan-then-execute system’s single wide approval, and a background agent’s single end-of-task pull request are not simply “more” or “less” safe than one another in a straight line — they sit at different, deliberately chosen points on a curve that flattens.

A standing lectern holding a clipped, multi-page printed plan, an unused pen resting beside a signature line it has not yet reached
Figure 4. A plan-then-execute system asks for one large approval before any step runs, trading many small checkpoints for a single wide one taken up front.Image prompt and art direction by Brecht Corbeel; image generated to that direction.

What persists: memory as a designed surface

Whether a system remembers anything at all between sessions, and who is responsible for what it remembers, is its own axis, and the approaches above answer it in ways that track their granularity choices more than their vendor. Claude Code documents two separate mechanisms rather than one: CLAUDE.md files, which are instructions a person writes and which load from up to four scopes — an organization-wide managed policy file, a personal ~/.claude/CLAUDE.md, a project file shared through version control, and a personal, gitignored local file — concatenated in that order rather than overridden, so a project instruction is read after a user instruction, not instead of it [5]. Alongside that sits “auto memory,” notes the model writes for itself — about the user’s role and preferences, corrections it has been given, ongoing project context it cannot derive from the code, and pointers to outside systems — stored per project and capped to the first 200 lines or 25 kilobytes read at the start of every session, on the reasoning that a memory file a person cannot audit at a glance is a memory file that will eventually be wrong without anyone noticing [5].

Inline copilots, by contrast, typically carry only a single static instructions file and whatever fits in the open editor buffer, because their unit of interaction never spans a whole session in the same sense. Autonomous background agents sit in between: because each task usually spins up a fresh sandboxed environment, what persists from one task to the next is mostly the state of the git repository itself — commits, branch history, code comments — rather than a dedicated memory file the agent maintains on its own initiative, which means teams using these tools most often end up encoding durable context the same place Claude Code does, in a checked-in instructions file the agent is configured to read at the start of each task.

A shared shelf of scope-labelled ring binders reachable from every desk, one binder half drawn out of its slot with its spine label only partly visible
Figure 5. What persists between sessions is itself a designed surface: a layered set of instruction files any desk can read, not a side effect of memory.Image prompt and art direction by Brecht Corbeel; image generated to that direction.

Delegation, outside tools, and where verification actually happens

Two further axes separate these approaches sharply, and then, unexpectedly, converge.

Delegation is where Claude Code’s design diverges most from the single-agent pattern common elsewhere. A subagent runs in its own context window, with its own system prompt and its own restricted tool list, started either automatically — when the model judges that a task’s verbose intermediate output would otherwise flood the main conversation — or explicitly by name; it can run in the foreground, blocking the main session, or in the background with a reduced tool set while the primary conversation continues [3]. Reaching outside the model’s own tool set entirely is handled by a separate mechanism, the Model Context Protocol, an open standard that lets Claude Code connect to “hundreds of external tools and data sources” — issue trackers, databases, monitoring systems — so that the model acts on a system directly instead of working from whatever a person has pasted into the conversation [4]. Both of these are deliberate widenings of what a single session can reach, and both raise the same question independent security research has been pressing on this whole category of tool: once an agent can read private data, is exposed to content it did not write itself, and has some way to communicate externally, those three conditions together are enough for a single piece of poisoned content to redirect what it does — a pattern one widely cited analysis names the “lethal trifecta” and singles out protocols like MCP specifically, because they “encourage mixing tools from different sources,” many supplying private data, many supplying untrusted content, and most supplying some path for a request to leave the system [10].

Verification is where the four approaches, for all their differences upstream, land in nearly the same place. Whatever gate an approach uses while work is happening — a per-action ask, a hook that blocks deterministically, a single plan sign-off, or nothing at all until a background task reports back — production use of every family described here still funnels into a human review of a concrete artifact before anything ships: a diff in Claude Code, a pull request from Copilot’s coding agent or from Codex, a pull request from Devin. GitHub’s documentation is explicit that human review remains built into the coding agent’s workflow regardless of how autonomously the session itself ran [6]. The interesting comparative fact is not that one approach verifies and another does not; it is that the industry has converged on roughly one choke point — a reviewable, revertible change under version control — even while disagreeing sharply about how much is allowed to happen automatically before a human ever sees that artifact.

A dispatch desk with routing slips travelling along a small rail toward labelled pigeonholes, and a rubber date-stamp caught descending onto a folder in the foreground just before it reaches the outbox
Figure 6. Delegated work and outside connections both pass back through one shared checkpoint before anything is called finished, whichever desk the work started at.Image prompt and art direction by Brecht Corbeel; image generated to that direction.

What changes for a software team

None of the architectural differences above determine how well a team actually does with any of these tools, and the largest available study of real-world adoption says so directly. DORA’s 2025 State of AI-assisted Software Development report, drawing on survey and interview data gathered from thousands of software professionals, describes AI’s primary effect on a team as that of an amplifier: it magnifies the strengths of an organization that already has clear workflows and a solid internal platform, and just as reliably magnifies the dysfunction of one that does not, with the report’s stated conclusion that the greatest returns come “not from the tools themselves” but from workflow clarity and platform quality that predate the tool’s arrival [11]. Read against everything above, that finding lines up with the architecture: a settings.json a team commits together, a CLAUDE.md a team maintains together, and a review gate a team already enforces are exactly the surfaces this article has been describing — they are just as load-bearing, and arguably more so, than which of the four broad approaches a team adopts first. A team that has none of those in place is unlikely to be rescued by choosing the “right” agentic tool, and a team that has all of them can likely make more than one of these approaches work.

Where this comparison breaks down

It is worth being explicit about the limits of everything above, because the temptation to compress this into a single ranked list is strong and the evidence does not support it. The SWE-bench figures cited here come from two different points in time, evaluated under two different scaffolds, one of them run by the vendor being described; they cannot be used to rank Devin against Claude Code, Copilot’s coding agent, or Codex, none of which is quoted against the same dated, independently reproduced benchmark run in this article. GitHub’s documented 59-minute execution cap and one-branch-one-PR constraint are specific to Copilot’s coding agent and do not generalize to “background agents” as a category. And the granularity trade-off model above is a simplification — it treats each tool call’s risk as independent and identically distributed, which is not true in practice, since some actions (a destructive shell command) carry far more downside than others (a file read) and a fixed per-checkpoint cost cc ignores that reviewing a whole plan is cognitively a different task from reviewing one command. The model is useful for seeing the shape of the trade-off, not for computing an actual number for any real team.

Predictions, with the observations that would falsify them

These are forecasts, separated clearly from the sourced material above. Horizon: August 2029.

One. The sharp line between “ask before every action” and “approve one plan up front” will blur into a single adjustable granularity setting inside individual tools, rather than remaining the boundary between separate product categories. Disconfirmed if the major vendors in 2029 still ship per-action approval and plan-then-execute as mutually exclusive products with no shared control surface.

Two. The convergence already visible at the verification stage will hold and strengthen: a required, reviewable diff or pull request will remain the near-universal final gate even as more of what happens upstream of it becomes unattended. Disconfirmed if a widely adopted agentic coding tool ships with autonomous merging as its default, unreviewed by a person, and that becomes the norm rather than a niche configuration.

Three. Cross-tool memory conventions — the pattern already visible in a competitor’s file being read by Claude Code’s own initialization command — will keep spreading, so that a project’s persistent instructions become readable by more than one vendor’s agent rather than locked to whichever tool wrote them first. Disconfirmed if, by the horizon date, the leading tools’ memory formats remain mutually unreadable with no cross-tool convention in wide use.

What to take away

Four questions sort this field better than any single benchmark can: who authorizes an action, what persists, how work gets delegated, and what gets checked before it counts as done. Claude Code answers with continuous, fine-grained, hookable approval and a layered written memory; inline copilots answer by folding approval into the keystroke and carrying almost no durable memory of their own; autonomous background agents answer by moving the entire loop off-machine and collapsing approval to one decision at assignment; plan-then-execute systems answer by moving that one decision earlier, to the plan itself, before any of it runs. None of these is simply safer or better than the others in the abstract — the granularity trade-off makes that explicit, and the review gate every approach converges on regardless is the strongest evidence that the real skill is not picking the right architecture but building the workflow around whichever one a team has chosen.