Level 1
Model call
A prompt and response may be enough for classification, extraction, rewriting, or structured generation.
Architecture guides
Use the least autonomous system that completes the job
An agent is not the default architecture. Start with a direct model call, retrieval, or a deterministic workflow; add planning and tool choice only when the task genuinely requires them.
Architecture ladder
Each step adds capability and operating burden. Move upward only when the simpler level cannot satisfy the acceptance criteria.
Level 2
Retrieval
Use indexed evidence when answers must be grounded in a controlled corpus and no external action is needed.
Level 3
Workflow
Use deterministic steps when the process and transitions are known, even if individual steps use a model.
Level 4
Agent
Use a bounded agent when the system must choose tools, adapt a plan, inspect results, and decide the next action.
Guides in this section
Evaluate AI Agents Before Deployment
Build a release gate from representative tasks, authority tests, failure injection, external outcome checks, and predeclared thresholds.
Read guideHuman Approval Patterns for AI Agents
Place durable, scoped approval at consequential actions without turning every model step into manual work.
Read guideAI Agent Observability
Define stable events, trace relationships, acceptance metrics, privacy controls, and release evidence for production agents.
Read guideAgent Framework Selection Checklist
A weighted rubric for deciding whether you need a framework, workflow engine, agent SDK, or custom loop.
Read guideRAG vs Agents
Choose retrieval when the job is grounded answering; add an agent only when the job requires controlled action and iteration.
Read guideLangChain Alternatives
Apply the architecture criteria to current production framework options.
Read guideA production architecture needs four contracts
Outcome contract
Define what the user accepts, which errors block release, and which examples represent ordinary, edge, and adversarial work. Without this contract, framework selection and evaluation become subjective.
Authority contract
List data, tools, targets, credentials, budgets, and actions. The application enforces this boundary; retrieved text and model output cannot expand it.
State contract
Specify what persists, how work resumes, which side effects are idempotent, and how versions affect pending tasks. State design determines whether recovery is safe.
Evidence contract
Connect task, retrieval, model, policy, approval, tool, and validation events. Operators need an explanation grounded in recorded actions, not a model-generated summary of itself.
Use RAG vs agents to select the architecture level, the framework checklist to compare runtimes, and the observability guide to make the running system accountable.
Choose the architecture from observable task signals
The deciding factor is not how sophisticated the request sounds. It is where uncertainty exists and what authority the system needs. Separate uncertainty about information from uncertainty about action: retrieval can resolve the first, while an agent may be justified only for the second.
| Architecture | Strong fit signal | Warning sign |
|---|---|---|
| Direct model call | One transformation, no external action, no durable state, and an output that can be validated immediately. | Adding memory, planning language, or tool routing without a measured need. |
| Retrieval | The answer must be supported by a controlled corpus and the main uncertainty is finding the right evidence. | Allowing retrieved instructions to change permissions or using citations that do not support the claim. |
| Deterministic workflow | The sequence is known, exceptions can be enumerated, and model judgment is limited to specific steps. | Hiding business rules in prompts or retrying external writes without idempotency. |
| Bounded agent | The next useful action depends on observed results and cannot be fully enumerated in advance. | Giving broad tools, open-ended loops, or approval authority before evaluation proves they are necessary. |
A useful design review writes the lowest sufficient level first, then documents the failing requirement that forces a move upward. For example, retrieval plus a fixed approval workflow may answer policy questions and create a draft ticket without requiring a general-purpose agent. Conversely, incident investigation across changing tools may require bounded planning, but destructive remediation can remain a separate approved workflow.
Define production readiness before implementation
Create a representative evaluation set before selecting a runtime. Include ordinary requests, missing evidence, conflicting instructions, permission boundaries, unavailable tools, malformed outputs, and interrupted runs. For each case, write the accepted result, prohibited actions, maximum intervention, and evidence an operator needs to diagnose failure.
Release criteria should cover more than response quality. Measure task acceptance, unsupported claims, tool-selection errors, policy denials, duplicate side effects, recovery success, human correction, tail latency, and total cost. Segment results by task type and risk: a strong average can conceal a dangerous failure rate on the few cases that send messages, alter records, or spend money.
Set an autonomy budget in addition to a financial budget. Limit steps, elapsed time, tool calls, destinations, records touched, and actions awaiting approval. When a limit is reached, the system should stop with a structured explanation and resumable state. Quietly extending the loop is not graceful recovery; it changes the authority granted by the original design.
Common architecture mistakes
Calling every multi-step prompt an agent
A label does not create autonomy. Identify who selects the next action, what state persists, and whether the system can affect anything outside its response.
Using memory as an unexamined transcript
Long histories increase cost and can preserve stale, sensitive, or malicious instructions. Store typed facts with provenance, purpose, retention, and update rules.
Letting the model define success
The component producing an answer should not be the sole judge of that answer. Use deterministic checks, source verification, external state, or human review.
Adding agents to repair an unstable process
Autonomy does not fix contradictory policies, missing ownership, inconsistent data, or undefined acceptance criteria. Stabilize the operating process first.
Observing tokens but not actions
Token counts cannot explain a duplicate payment, wrong destination, or approval bypass. Trace decisions, policy results, tool arguments, side effects, and validation.
Practical conclusion
Choose direct generation for a validated transformation, retrieval for evidence-bound answering, a workflow for known transitions, and an agent only for bounded decisions that depend on observed results. Keep irreversible actions outside the model’s unilateral control. This architecture ladder reduces cost and failure surface while making every later increase in autonomy an explicit, reviewable decision.