AI Agents and Architecture

An AI agent is a language model placed inside a loop that lets it act, not only answer. This area covers the parts that make that loop work: the architecture holding it together, the memory that carries context between turns, the tools that let it reach the outside world, and the controls that decide how far it may go on its own. Almost everything else in agent design is a variation on those four concerns.

It matters because the gap between a demo and a dependable system lives here. A model that answers well can still fail as an agent if it forgets what it already did, calls the wrong tool, or loops without ever finishing. People building agents and people buying them need the same vocabulary to describe where control sits, what the agent is allowed to decide, and who reviews the result before it becomes permanent.

Start here

How the pieces fit together

Start at the loop. A model receives a goal, decides on an action, an executor runs it, and the result comes back as new input. Everything else attaches to that cycle. Planning and task decomposition shape what the next action should be. Tools are the only way the cycle touches anything real. Memory and state decide what survives from one pass to the next, which is why an agent with tools but no memory keeps repeating itself. Orchestration sits one level above, deciding which agent runs when, and a supervisor is really just an agent whose tools are other agents. The trajectory is the recorded path through all of it.

Where autonomy actually comes from

Autonomy is not a property of the model. It comes from three separate choices made by whoever builds the system: how many steps the loop may take before stopping, which tools are wired in, and which actions require a person to approve them first. A capable model with read-only tools and an approval gate is a low autonomy system. A weaker model with write access to a payments endpoint and no gate is a high autonomy one. Reading autonomy off the model is the most common category error in this area, and it explains how two products described in nearly identical language can carry very different risk.

The order to learn it in

Read the loop before anything that decorates it. Once the cycle is clear, tool use is the natural next step, because it explains where an agent's real capability comes from. Memory follows, because it explains why a loop that works for three steps falls apart at thirty. Only then does multi-agent design make sense. Most teams reach for several agents too early, before a single agent with good tools and a clean state model has had a fair attempt, and they inherit coordination problems they never needed. Persona, planning style, and scheduling are refinements that assume the base loop already behaves.

What people get wrong

Three mistakes recur. The first is treating a longer prompt as a substitute for structure, when the real failure is that nothing carries state between steps. The second is assuming an agent behaves the same way twice. The surrounding system can be made deterministic, but the model's choices are probabilistic, so tests and guardrails have to tolerate variation rather than assert one exact path. The third is putting a human review step somewhere nobody looks. Review works when it sits on the specific action that is expensive to undo, and it turns into noise the moment it fires on everything.

Commonly confused

Autonomous Agent vs Agentic Workflow

A workflow follows a path someone designed in advance, while an autonomous agent chooses its own next step, which is why one is straightforward to test and the other has to be bounded.

Agent Orchestration vs Multi-Agent System

Orchestration is the coordination logic; a multi-agent system is the arrangement of agents that logic coordinates, and you can orchestrate a single agent.

Delegation vs Handoff

Delegation keeps ownership with the sender, who gets the result back, while a handoff transfers ownership and leaves the original agent out of the conversation.

Chain of Thought vs Agent Planning

Chain of thought is reasoning that happens inside one model call; planning produces a structure the system then acts on across many calls.

Every term in AI Agents and Architecture

Agent Architecture: Agent architecture is the arrangement of components that make an agent work: the model, the instructions, the tool layer, the memory and state stores, the control loop, and the guardrails. It describes how information flows between them and where each decision is made. Different architectures suit different tradeoffs between flexibility, cost, and predictability.
Agent Constitution: An agent constitution is a written set of principles that govern an agent's behavior, stating what it must always do, never do, and how to resolve conflicts between goals. In deployed systems it usually appears as durable instruction text and enforcement rules, distinct from the research method of training a model against a written set of principles.
Agent Framework: An agent framework is a class of software library or toolkit that provides the scaffolding for building systems in which a language model plans, calls tools, and acts over multiple steps. Typical components include a control loop, tool registration, state and memory handling, and observability hooks. Frameworks differ in how much control they take from application code.
Agent Loop: The agent loop is the repeating cycle at the center of every agent: assemble context, ask the model what to do, execute the chosen action, add the result to context, and repeat until a stopping condition is met. Stopping conditions include producing a final answer, hitting a step or budget limit, or requiring human approval.
Agent Memory: Agent memory is information deliberately retained across separate runs or conversations so an agent can recall facts, preferences, and past outcomes it was not given in the current request. It is stored outside the model, commonly in a database, vector index, or knowledge graph, and relevant pieces are retrieved and inserted into context before each step.
Agent Orchestration: Agent orchestration is the layer that decides which agent or step runs, in what order, with what inputs, and what happens when something fails. It covers routing, scheduling, retries, state passing, and concurrency limits. Orchestration is ordinary software rather than model output, which is what makes multi-step agent systems observable and recoverable.
Agent Persona: An agent persona is the defined role, voice, and scope given to an agent, covering what it is responsible for, how it communicates, and what it declines. It is expressed mainly in the system prompt and shapes both output style and behavior, because a clearly bounded role improves tool selection and reduces answers outside the agent's remit.
Agent Planning: Agent planning is the process by which an agent works out an ordered set of steps to reach a goal before, or while, carrying them out. Plans may be produced once at the start, revised after each observation, or replaced entirely when new information arrives. Planning quality is a common bottleneck, because a confident plan built on a wrong assumption fails at every step.
Agent Role: An agent role is the scoped function assigned to an agent within a system, defining what it is responsible for, which tools and data it may access, and what it must hand off to others. Roles are structural boundaries rather than descriptive labels, and they are most useful when access permissions actually follow them.
Agent Run: An agent run is one bounded execution of an agent from trigger to termination, containing every step, tool call, and intermediate result produced along the way. It is the standard unit for measuring cost, duration, and success, and it is what logging, replay, and evaluation systems are usually built around.
Agent Skill: An agent skill is a packaged unit of instruction that teaches an agent how to perform a particular kind of task, typically containing procedures, conventions, and references, and loaded into context only when relevant. Skills keep the base prompt small while making specialized know-how available on demand. The term is used differently across frameworks, so the details vary.
Agent State: Agent state is the information an agent carries through a single run: the conversation so far, intermediate results, the current plan, pending approvals, and progress markers. It lives in the runtime rather than inside the model, because a model holds nothing between calls. Durable state is what lets a run pause, resume, or be inspected after the fact.
Agent Swarm: An agent swarm is a multi-agent arrangement in which many peer agents work on a shared goal without a central controller, coordinating through local interaction or a shared medium rather than through a directing supervisor. Global behavior emerges from local decisions, which makes swarms flexible but harder to predict and audit than hierarchical designs.
Agent Trajectory: An agent trajectory is the complete recorded sequence of a single run: every model input, decision, tool call, result, and error, in order. It is the primary artifact for debugging, evaluation, and audit, because the final answer alone does not reveal how it was produced. Trajectories also show whether an agent took a sensible route, not just whether it finished.
Agentic AI: Agentic AI is an umbrella term for AI systems that pursue goals through multiple steps and actions rather than producing a single response. It describes an approach rather than a specific technology, covering anything from a model that calls one tool to a coordinated set of agents. The term is marketing-heavy and its boundaries are not agreed on.
Agentic Workflow: An agentic workflow is a defined sequence of work in which one or more AI agents carry out steps, with the surrounding structure specifying the order, the inputs, and the checkpoints. It sits between a rigid automation script and a fully open-ended agent, because the shape of the work is fixed while the content of each step is decided at runtime.
AI Agent: An AI agent is a software system that uses a language model to decide what actions to take toward a goal, then carries those actions out through tools such as APIs, databases, or a browser. Unlike a single question and answer exchange, an agent runs over multiple steps, observing results and adjusting until the goal is met or it stops.
Autonomous Agent: An autonomous agent is an AI agent that can start, continue, and finish work without a person approving every step. Autonomy is a spectrum rather than a switch, and it is usually bounded by explicit limits on which tools the agent may use, how much it may spend, and which actions require approval before they take effect.
Autonomy Level: Autonomy level describes how much an agent may do without human involvement, from suggesting an action, to acting with approval, to acting freely within limits and reporting afterward. There is no single industry standard scale, so specific levels are defined per system. Levels are usually assigned per action type rather than to the agent as a whole.
Blackboard Architecture: A blackboard architecture is a coordination pattern in which independent components read from and write to a shared, structured data store rather than calling one another directly. Each component watches the store for conditions it can act on, contributes its result, and lets other components build on that contribution, with a control mechanism deciding who acts next.
Chain of Thought: Chain of thought is the practice of having a language model produce intermediate reasoning steps before its final answer, rather than jumping straight to a conclusion. It measurably improves accuracy on multi-step problems such as arithmetic, logic, and planning. The written steps are an output of the model, not a transcript of how it actually computed the answer.
Checkpointing: Checkpointing is the practice of persisting an agent's state at defined points during a run so that execution can resume from the last saved point after an interruption, rather than restarting from the beginning. It underpins long running agents, human approval pauses, and recovery from process crashes or deployments.
Confidence Threshold: A confidence threshold is a cutoff value that determines whether an agent acts on a result automatically or routes it for review. It requires a confidence signal that actually correlates with correctness, and a language model's own stated confidence is a weak signal, since models are frequently confident and wrong.
Context Engineering: Context engineering is the practice of deciding what information occupies an agent's context window at each step, including instructions, retrieved documents, tool definitions, prior messages, and working state. It treats the context window as a scarce resource to be curated deliberately rather than as a buffer that accumulates everything the run has touched.
Delegation: Delegation is when one agent assigns a subtask to another agent and remains responsible for the outcome. The delegating agent frames the request, supplies the context the other will need, and receives a result to check and integrate. It differs from a handoff, where responsibility for the interaction transfers away and does not return.
Deterministic Replay: Deterministic replay is the reconstruction of a past agent run by replaying its recorded inputs, model outputs, and tool results in the original order, rather than re-executing the model and tools live. It makes a nondeterministic run reproducible for debugging, since the recorded outputs are fixed instead of being sampled again.
Deterministic vs Probabilistic Behavior: Deterministic behavior produces the same output for the same input every time, while probabilistic behavior samples from a distribution and may differ between runs. Language models are probabilistic by default, so agents built on them are not reproducible in the way ordinary software is. Reliable systems place deterministic code around probabilistic decisions rather than trying to remove the variability.
Escalation Policy: An escalation policy defines when an agent must stop acting on its own and route a decision to a person or a higher authority agent. It specifies the triggering conditions, who receives the request, what context accompanies it, and what the agent does while it waits, including whether the run pauses or continues on other work.
Event-Driven Agent: An event-driven agent starts in response to something happening, such as an incoming message, a webhook, a record changing, or a threshold being crossed. Its trigger carries the initial context, and its value is responding close to the moment the event occurs. Volume is unpredictable, so limits and deduplication matter more than in scheduled work.
Fallback Behavior: Fallback behavior is what an agent does when its intended path is unavailable, such as a failed tool, an unreachable model, an exhausted budget, or a result below a confidence threshold. A defined fallback makes failure predictable, whereas an undefined one leaves the agent to improvise, which commonly produces a fabricated answer.
Guard Condition: A guard condition is a check evaluated before an agent action is allowed to proceed, blocking it when the check fails. Guards are enforced by the runtime rather than by the model, which makes them effective against reasoning errors, ambiguous instructions, and injected instructions alike.
Handoff: A handoff is the transfer of an in-progress task or conversation from one agent to another, or from an agent to a person, along with the context needed to continue. Unlike delegation, responsibility moves and typically does not return. Handoffs are common in support routing, escalation, and specialist agent designs.
Human in the Loop: Human in the loop describes an agent design in which a person reviews, approves, corrects, or supplies input at defined points before the system proceeds. It is typically applied to actions that are costly or hard to reverse, such as sending external messages, changing records, or spending money. The pause is enforced by the runtime, not requested by the model.
Multi-Agent System: A multi-agent system is one in which several AI agents, each with its own instructions, tools, and scope, work on parts of a larger goal and exchange information. Agents may run in sequence, in parallel, or under a coordinator. The design trades the simplicity of one agent for specialization, isolation of context, and parallel progress.
Plan And Execute Pattern: Plan and execute is an agent design in which a planning step produces an explicit multi-step plan up front, and a separate execution step carries out those steps in order, optionally replanning when a step fails. It contrasts with interleaved designs that decide only the next single action after observing the previous result.
ReAct Pattern: ReAct, short for reasoning and acting, is an agent pattern in which the model alternates between writing out a thought about what to do next and taking an action such as a tool call, then observing the result before thinking again. Introduced in a 2022 research paper, it became the default shape for tool-using agents.
Reasoning Model: A reasoning model is a language model trained to spend additional computation on internal deliberation before answering, typically producing extended intermediate reasoning that may be hidden from the caller. Several major providers offer such models alongside faster general-purpose ones, often with a setting controlling how much reasoning effort to spend. They cost more and respond more slowly.
Recursion Limit: A recursion limit is the maximum nesting depth allowed when agents create other agents, or when a graph based agent revisits the same node. It prevents unbounded delegation chains where each agent spawns another, and it is enforced by the runtime rather than by any single agent's judgment.
Reflection Pattern: The reflection pattern is an agent design in which a first pass output is fed back for explicit criticism, and the criticism is then used to produce a revised output. The critique may come from the same model in a separate call, a different model, or an automated check such as a compiler, test suite, or validator.
Scheduled Agent: A scheduled agent runs on a time-based trigger such as an hourly, daily, or weekly schedule, rather than in response to a request. Nobody is present when it starts, so it must determine its own context, decide whether there is anything to do, and route its output somewhere a person will see it later.
Self-Consistency: Self-consistency is a technique that samples several independent reasoning paths for the same question and selects the answer that appears most often, rather than trusting a single generated chain. It exploits the observation that correct answers tend to be reached by many different valid routes, while errors are more scattered across sampled outputs.
Shared Scratchpad: A shared scratchpad is a mutable working area that several agents, or several steps of one agent, can read and write during a task. It holds intermediate notes, partial results, and open questions that would otherwise be lost between steps, and it is normally discarded when the task ends rather than persisted as long term memory.
Stateful vs Stateless Agent: A stateless agent handles each request independently, retaining nothing between invocations, while a stateful agent carries information forward across steps or sessions through stored conversation history, memory, or working state. The distinction determines whether identical inputs produce comparable behavior and how much infrastructure a deployment requires.
Step Budget: A step budget is the maximum number of actions, usually model calls or tool invocations, that an agent may take within a single run. It bounds the cost and duration of any one execution, converting a potentially unbounded loop into a run whose worst case is known before it starts.
Subagent: A subagent is an agent instance created by another agent to carry out a bounded part of a larger task and return a result to its caller. It usually receives its own instructions, tool set, and context window, and its intermediate steps stay hidden from the parent, which observes only the returned output.
Supervisor Agent: A supervisor agent is an agent whose job is to coordinate other agents rather than to do the work itself. It interprets the goal, decides which specialist should act next, passes along the necessary context, and decides when the overall task is complete. It is the most widely used arrangement in multi-agent systems.
Task Decomposition: Task decomposition is breaking a large or vague request into smaller, concrete subtasks that can be executed and checked one at a time. In agent systems the split may be written by a developer, produced by the model at runtime, or a mix of both. Good decomposition produces subtasks with clear inputs, clear completion criteria, and few dependencies.
Termination Condition: A termination condition is the rule that ends an agent's execution loop. It may be satisfaction, meaning the agent judges the goal complete, or exhaustion, meaning a limit on steps, time, cost, or consecutive failures was reached. Every autonomous loop needs at least one exhaustion condition, because satisfaction alone can never be guaranteed to trigger.
Tool Error Recovery: Tool error recovery is how an agent responds when a tool call fails, times out, returns unexpected data, or is rejected. It covers whether the failure is surfaced to the agent as an observation, how the agent is expected to react, and what limits prevent it from retrying the same failing call indefinitely.
Tool Selection: Tool selection is the decision an agent makes about which available tool, if any, to invoke at a given step. It depends on the tool descriptions the agent can see, the task at hand, and the state of the current run, and its accuracy degrades as the number of similar or overlapping tools grows.
Tool Use: Tool use is an agent's ability to invoke external functions such as search, database queries, file operations, or third-party APIs, and to incorporate the results into its next decision. The model does not run the tool itself. It emits a structured request naming the tool and its arguments, and the surrounding runtime executes it and returns the output.
Tree Of Thoughts: Tree of thoughts is a reasoning method in which a model generates several candidate intermediate steps at each stage, evaluates how promising each one is, and searches the resulting tree with strategies such as breadth first or depth first exploration, including backtracking. It generalizes single path step by step reasoning into a deliberate search over alternatives.
Wake Condition: A wake condition is the rule that causes an idle agent to begin a run. It may be a schedule, an incoming event such as a message or webhook, a state change detected by polling, or a threshold crossing in monitored data, and it determines both how promptly an agent responds and how much it costs while nothing is happening.

Back to the AI Glossary