Five drawers, one question
Every architecture described in this article is a different answer to the same design question: what, mechanically, decides what an artificial agent does next? The question is older than deep learning and older than the word “agent” as it is used in AI research today. It was already being answered, in hardware and hand-written logic, by a wheeled robot at a California research institute in the late 1960s — slowly, and only inside a world its planner had been told about in advance.
This is a history, not a survey of current products, and it is organized around five successive, dated answers to that one question: a symbolic theorem-proving planner that computed an entire sequence of actions before the first one executed; a learned value function that mapped situations to actions through reward rather than logic; a pretrained language model interleaving reasoning and action inside a single prompted loop; that loop split explicitly into a planning role and an executing role, sometimes multiplied into several cooperating agents; and, most recently, the whole arrangement shipped as maintained, versioned infrastructure by the labs that build the underlying models. Each stage is dated to a specific paper or product announcement, not to a vague decade, because the actual chronology is more useful for judging what comes next than the flattering just-so story usually told about it.
Two points run through every stage and are worth naming before the history starts. First, an agent architecture is defined by where its “next action” decision physically lives — in a search over symbolic operators, in trained weights, in a prompt, in a second model call, in a library function — not by how impressive its outputs read. Second, every stage below eventually broke against the same wall: single-shot competence is cheap to demonstrate and expensive to sustain over a long, consequential task. What differs, stage to stage, is what the field built to push that wall back.
Origins: a planner that proved its plan before acting
The clearest starting point in the documented record is Shakey, a mobile robot built at SRI International’s Artificial Intelligence Center and worked on from 1966 to 1972. SRI’s own account describes it as the first robot able to “perceive its surroundings, infer implicit facts from explicit ones, create plans, recover from errors in plan execution, and communicate using ordinary English” [1]. It moved through a small set of rooms, and by the account of the institution that built it, was capable of “tasks that required planning, route-finding, and the rearranging of simple objects” [1].
The planner underneath Shakey was STRIPS — the Stanford Research Institute Problem Solver — described by Fikes and Nilsson in a 1971 paper in the journal Artificial Intelligence. STRIPS represented the world as a set of predicate-logic statements, represented each available action as an operator with a precondition list, a delete list, and an add list, and used a resolution theorem prover guided by means-ends analysis to search for a sequence of operators connecting the current state to a goal state [2]. Formally, for a state
This is worth writing out because it is a real, load-bearing assumption rather than notation for its own sake: everything about the world not explicitly named in an operator’s delete or add list is assumed to persist unchanged. Without that assumption, an operator’s author would have to state every fact that stays true across every action — combinatorially unworkable. With it, STRIPS could plan efficiently inside a closed, fully modeled world. The same assumption is also the reason planners of this kind struggled the moment they left one: any real-world consequence the plan’s author forgot to encode simply did not exist for the planner, and a divergence between the model and the actual room meant replanning from scratch rather than adapting mid-execution.
The architecture this establishes is worth naming precisely, because later stages are best understood as departures from it. Shakey’s loop was sense, build a symbolic model, plan an entire action sequence against that model, then execute it largely open-loop. The “intelligence” of the system lived in two places: the completeness of the hand-written operator library, and the correctness of the search that assembled operators into a plan. Nothing about the architecture learned from experience; every capability had to be anticipated and encoded by a person before the robot could exhibit it.
The reinforcement-learning era: agents that learn what to do
The next architectural break replaced the hand-written operator library with something the agent acquired through trial, error, and a numeric reward signal, rather than something a person specified in advance. Sutton and Barto’s textbook formalizes the setting that every reinforcement-learning agent since has used: an agent and an environment exchanging a state, an action, and a scalar reward at each discrete time step, with the agent’s goal defined as maximizing cumulative reward rather than satisfying a hand-specified goal predicate [3]. The canonical learning rule for estimating the value of taking action
That equation is the real content of the RL-era architecture, not decoration: the agent’s decision rule is now a function fit incrementally from its own experience, and the “plan” a STRIPS operator library encoded explicitly is replaced by whatever policy falls out of maximizing
Mnih and colleagues’ Deep Q-Network, published in Nature in 2015, showed this rule scales when the value function is a deep neural network trained directly on raw pixels rather than a hand-engineered state representation. Trained separately on 49 different Atari 2600 games “using the same algorithm, network architecture and hyperparameters,” the agent achieved performance the paper describes as comparable to that of a professional human games tester, and reported that it was “able to surpass the performance of all previous algorithms” on the benchmark [4]. Silver and colleagues’ AlphaGo, also published in Nature, combined a policy network and a value network — trained first by supervised learning on human expert games and then by reinforcement learning through self-play — with Monte Carlo tree search, and used that combination to defeat the human European Go champion, Fan Hui, five games to zero in October 2015, the first time a program had beaten a professional human on a full-size board without a handicap [5, 6].
The architectural point to take from this era is specific. An RL-era agent’s competence lived in trained weights fit to one environment; a DQN trained on one Atari game could not play a second without retraining, and AlphaGo’s networks were fit to Go specifically, not to games in general. This is a genuinely different mechanism from Shakey’s — nothing here is a symbolic plan a person could read — but it shares Shakey’s central limitation in a new form: capability was baked into a fixed artifact, whether an operator library or a weight matrix, before deployment, and the artifact did not generalize to problems its training had not anticipated.
Language models start acting
By the early 2020s, large pretrained language models could produce fluent, multi-step reasoning text, but that reasoning had no way to touch the world: a model asked to answer a factual question could not check a reference, and a model narrating a plan could not verify that any step in it was still true. Two papers from 2022 and 2023, addressing this from opposite directions, remain the documented root of essentially every LLM-based agent shipped since.
Yao and colleagues’ ReAct interleaves a model’s reasoning trace with concrete actions and the observations those actions return, rather than treating reasoning and acting as separate skills studied in isolation. The paper’s own account of why this helps is precise: reasoning traces let the model “induce, track, and update action plans as well as handle exceptions,” while actions let it “interface with external sources, such as knowledge bases or environments, to gather additional information” [7]. On question answering and fact verification, grounding the model’s reasoning in lookups against a simple Wikipedia API reduced the hallucination and error propagation that the paper reports as prevalent in chain-of-thought reasoning run without any external check; on two interactive decision-making benchmarks, ALFWorld and WebShop, ReAct outperformed imitation-learning and reinforcement-learning baselines by an absolute success rate of 34 and 10 percentage points respectively, using only one or two in-context examples rather than task-specific training [7]. That last detail marks the real break from the previous section: an RL-era agent’s competence lived in weights specific to one environment; a ReAct-style agent’s competence lives substantially in the prompt, and the same underlying model is redirected to a new environment by changing the text around it rather than by retraining it.
Schick and colleagues’ Toolformer approached the same problem from the training side. Rather than imposing a reasoning-then-acting pattern at inference time, a model is taught, through a self-supervised procedure applied to its own generated continuations, when to call an external API — a calculator, a search engine, a translator, a calendar — what arguments to pass, and how to fold the result back into what it produces next, learning this from as few as a handful of demonstrations per tool [8]. Where ReAct is a control pattern imposed on a fixed model, Toolformer changes what the model itself has learned to do; most later agent frameworks combine both ideas, a prompted loop structure wrapped around a model at least partly trained to use tools competently.
A third piece was adopted quickly once agents were reliably calling tools and reporting results back: some mechanism for learning across attempts without retraining the underlying model. Shinn and colleagues’ Reflexion sits a lightweight memory on top of a ReAct-style loop — after a failed trial, the agent produces a verbal, textual reflection on why it failed, stores that reflection in an episodic buffer, and carries it into the next attempt, with no gradient update to the model at all [9]. On one coding benchmark, the paper reports a Reflexion agent reaching 91 percent pass@1 accuracy on HumanEval, compared with 80 percent reported for GPT-4 without the reflection loop at the time [9]. That is a single benchmark result from one paper, not a general law about self-reflection, and this article treats it as such — but the pattern it established, treating a failed attempt as reusable data rather than a discarded rollout, persists in every architecture described below.
By late 2023, the field had reassembled something close to Shakey’s sense-plan-act loop, except the planner and the actor were now the same prompted model, called repeatedly, with a growing transcript standing in for both working memory and plan. What it still lacked was any explicit separation of roles: one model call did the thinking, the acting, and the reflecting, all inside a single, ever-growing context window.
Planner, executor, supervisor: the loop splits into roles
A single loop with a long transcript degrades in a specific, documented way: on tasks requiring more than a handful of steps, a model re-deciding its entire strategy at every turn tends to lose track of the plan it was implicitly pursuing, repeat finished work, or drift from the original goal. The field’s response through 2023 and 2024 was to make the planning step explicit again — not by returning to STRIPS-style theorem proving, but by asking a model to produce a plan as a distinct, inspectable artifact before execution begins.
Wang and colleagues’ Plan-and-Solve prompting states this as a deliberate technique: rather than a single “let’s think step by step” instruction, the model is prompted in two explicit stages, “first, devising a plan to divide the entire task into smaller subtasks, and then carrying out the subtasks according to the plan” [10]. The paper’s stated motivation identifies three distinct failure modes in zero-shot chain-of-thought reasoning — calculation errors, missing reasoning steps, and semantic misunderstanding — and targets the missing-step failure directly by separating planning from execution [10].
Anthropic’s own applied guidance from December 2024 generalizes the same split from a single prompt into a system of separate model calls. It distinguishes a workflow, where “LLMs and tools are orchestrated through predefined code paths,” from an agent, where the model “dynamically directs its own processes and tool usage,” and names orchestrator-workers as one recurring pattern: “a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results,” used where subtasks cannot be predicted in advance, such as coordinated changes across several files [12]. That is a planner/executor architecture in the STRIPS-era sense — a phase deciding what should happen, separated from a phase making it happen — rebuilt from language-model calls rather than symbolic operators.
Multi-agent frameworks generalized the idea again, from one orchestrator directing interchangeable workers to a genuine society of differently specialized agents. Wu and colleagues’ AutoGen, released by Microsoft Research in 2023, frames the unit of composition as a conversation: customizable, conversable agents combining language models, human input, and tools, coordinated through natural-language or code-based patterns a developer defines rather than hard-codes [11]. LangChain’s LangGraph took a narrower, named version of the same idea: a supervisor pattern, in which “an agent supervisor… is responsible for routing to individual agents,” each keeping its own independent scratchpad and sharing only a final response back to the supervisor rather than every intermediate step [17]. The stated rationale is architectural: grouping tools and responsibilities gives better results because “an agent is more likely to succeed on a focused task than if it has to select from dozens of tools” [17] — the same argument, in different words, that motivated bounding STRIPS operators into a small, well-defined action set half a century earlier.
It is worth stating plainly where the field disagrees, rather than picking a side. Anthropic’s own guidance, in the same document that describes orchestrator-workers, warns against reaching for multi-agent complexity by default, recommending the simplest architecture that works, because added autonomy trades latency and cost for flexibility and creates room for compounding error [12]. Frameworks built explicitly around multi-agent decomposition, AutoGen and LangGraph among them, start from something closer to the opposite prior: that decomposing a task across specialized agents is usually the more robust default beyond a narrow task. Neither position is a settled empirical finding in the sourcing gathered here; both are documented, differing engineering judgments from teams that build agents for a living, and “more agents” is a design decision with a real cost, not an automatic upgrade.
Official agent SDKs: the loop becomes vendor-maintained infrastructure
Every architecture through 2024 — ReAct loops, Toolformer-trained tool use, Reflexion-style memory, orchestrator-worker workflows, AutoGen and LangGraph multi-agent systems — was assembled by an individual team or an open-source project on top of a general-purpose model API. The tool-calling contract, the sandboxing, the permissioning, and the multi-agent handoff logic were each reinvented per project. The most recent shift in this history is not a new decision-making mechanism; every agent below is still a descendant of the ReAct loop. What changed is that the surrounding infrastructure became vendor-shipped, versioned, and documented rather than bespoke.
The first documented step toward standardizing the connective tissue, rather than the loop itself, was Anthropic’s Model Context Protocol, announced on November 25, 2024. Anthropic’s stated reasoning was explicit about the problem it was built to solve: “even the most sophisticated models are constrained by their isolation from data — trapped behind information silos and legacy systems,” and “every new data source requires its own custom implementation, making truly connected systems difficult to scale” [13]. MCP standardizes one specific seam, how an agent host talks to a tool or data source, so a connector written once works with any compliant agent rather than one specific framework.
OpenAI followed in March 2025 with a first-party toolkit aimed at the harness itself: an open-source Agents SDK, described in its own repository as “a lightweight yet powerful framework for building multi-agent workflows,” deliberately provider-agnostic across OpenAI’s own APIs and more than a hundred other models [16]. Its named primitives read as a direct, productized descendant of the previous section’s patterns: agents defined by instructions and tools, handoffs between agents — the supervisor pattern as a documented API call rather than a hand-written routing function — guardrails, persistent sessions, and built-in tracing for inspecting what a multi-agent run actually did [16].
Anthropic’s equivalent arrived on September 29, 2025, alongside Claude Sonnet 4.5, framed by the company as an internal tool made external: “We’re also giving developers the building blocks we use ourselves to make Claude Code. We’re calling this the Claude Agent SDK” [14]. Current documentation describes it as running “the same tools, agent loop, and context management that power Claude Code,” and lists subagents — the ability to “spawn specialized agents for focused subtasks” — alongside hooks, MCP connectivity, and fine-grained permissions as built-in capabilities [15]. Where AutoGen and LangGraph required a developer to assemble a multi-agent topology from general-purpose parts, a subagent call in the Claude Agent SDK and a handoff call in the OpenAI Agents SDK are now documented, single function calls.
What makes this a genuine architectural stage, and not merely a rebrand of the same ReAct loop, is what moved from research paper to maintained default: retry and error handling, per-tool permission scoping, session persistence across a long task, and the multi-agent handoff pattern itself are no longer something every team re-derives from the papers cited above. They are arguments to a constructor. That is a real change in where the field’s accumulated engineering judgment lives — not in a paper a new team must have read, but in a library a new team imports.
Two predictions, and what would falsify them
These are forecasts, kept separate from the sourced history above. Horizon: August 2029.
One. The vocabulary now shipping in production agent SDKs — agents, tools, handoffs or subagents, sessions, guardrails — will keep converging across vendors rather than diverging, the way early web frameworks converged on request, response, and middleware after a period of incompatible reinvention. The assumption behind this forecast is that developers building on more than one model provider are already common, and a shared vocabulary lowers the cost of switching or combining providers. Disconfirmed if, by August 2029, the major vendors’ agent SDKs still use structurally incompatible primitives for the same three concepts — defining a tool, handing off between agents, and persisting a session — such that a system built on one cannot be described in the other’s terms without a full rewrite.
Two. Some form of explicit, checkable plan, closer in spirit to a STRIPS operator sequence than to a free-text chain-of-thought, will re-enter mainstream agent architectures as a reliability measure specifically for the highest-consequence actions, even though the field spent 2022 through 2025 moving away from hand-authored symbolic planning. The assumption is that natural-language plans are hard to verify mechanically before execution, and that as agents are given higher-stakes, harder-to-reverse actions, teams will want a plan representation a separate piece of software can check rather than only read. Disconfirmed if production agent frameworks in 2029 still route irreversible actions through the same unstructured free-text plan representation used for everything else, with no distinct, machine-checkable planning step reserved for high-consequence actions.
What has actually changed, and what has not
Read as a sequence of dated artifacts rather than a marketing narrative, the history is narrower than “AI agents got smarter.” Five architectures answered one question — what decides the agent’s next action — with five different mechanisms: a resolution theorem prover reasoning over hand-written operators; a value function fit by trial, error, and reward; a pretrained language model interleaving reasoning and action in a single prompted loop; that same loop split explicitly into a planning role and an executing role, sometimes multiplied across several specialized agents; and, most recently, that entire arrangement packaged as versioned, vendor-maintained infrastructure.
None of these five replaced the one before it outright. Reinforcement learning still trains the reward models behind today’s language-model agents, and a language-model agent asked to plan a multi-file code change is still, underneath the SDK, running something a 2023 ReAct paper would recognize. What moved, at each stage, was not the ambition but the mechanism — and the honest way to evaluate a new agent claim is still to ask the same question this article asked of every stage: what, specifically, decides what happens next, and on what evidence would that decision have been different.
