AI Agent Security Checklist Before Production

Eight control areas

Area	Release requirement
Identity	Every run has an authenticated user or service identity and a recorded owner.
Data	The agent receives only data allowed for the current user, task, and retention policy.
Tools	Each tool has an allowlist, schema validation, timeout, narrow credential, and audit record.
Instructions	External content is treated as untrusted data and cannot grant new authority.
Outputs	Model output is validated before code, queries, URLs, files, or messages reach another system.
Approvals	Irreversible, external, privileged, or high-impact actions require review of exact arguments.
Runtime	Runs have step, time, token, retry, tool, and spend ceilings.
Monitoring	Operators can reconstruct the request, state, model calls, tool actions, approvals, and result.

1. Establish identity and ownership

Every run needs an authenticated principal, a workflow owner, and an environment. Do not let the model choose which user, tenant, account, or organization it represents. Service identities should be separate by environment and purpose so a compromised development workflow cannot access production resources.

2. Minimize data exposure

Classify inputs before sending them to a model or tool. Retrieve data through the same authorization rules used by the application. Redact secrets and unnecessary personal information, and set retention rules for prompts, traces, screenshots, files, and tool responses.

3. Treat external instructions as untrusted

Web pages, email, documents, retrieved passages, issue comments, and tool output may contain instructions intended to manipulate the agent. Separate system policy from external content. A document may provide evidence, but it cannot authorize a new tool, reveal a secret, change the target account, or disable review.

4. Build a tool gateway

Route tool calls through application code that validates the tool name, arguments, target, permission, and approval requirement. Use narrow credentials rather than a shared administrator token. Add timeouts, rate limits, network restrictions, safe file paths, and idempotency keys where side effects are possible.

5. Validate model output

Model output is untrusted input to the next component. Validate structured data against a schema. Parameterize database queries. Restrict URLs and file paths. Never pass model-generated text directly to a shell, template engine, SQL interpreter, browser navigation, or outbound message without context-specific checks.

6. Require approval for high-impact actions

Human review must display the exact action, target, content, cost, and relevant evidence. “Approve task” is too broad if the agent can later change the arguments. Require a new approval after a material change, and expire approvals so they cannot be replayed.

7. Constrain runtime and cost

Set maximum steps, duration, tokens, retries, concurrent tools, and spend. A retry must distinguish a safe read from a side effect. Stop when evidence is insufficient, when policy validation fails, or when repeated attempts do not change the result.

8. Preserve traces and incident controls

Store trace IDs linking the request, retrieved evidence, model calls, state changes, tool arguments, approvals, responses, and final outcome. Protect traces because they may contain sensitive data. Provide a kill switch, credential revocation procedure, queue pause, and way to identify affected runs after an incident.

Adversarial release tests

Place a malicious instruction inside a retrieved document and confirm it cannot expand permissions.
Ask the agent to send data to an unapproved domain or account.
Return malformed tool output containing code, paths, or URLs.
Timeout a tool after it may have completed a side effect and verify idempotent recovery.
Attempt to access another user or tenant’s data through retrieval and tool arguments.
Exhaust the step or spend budget and confirm the workflow stops safely.
Change an approved action’s target and confirm a new approval is required.
Reconstruct a failed run using stored evidence without relying on the model’s explanation.

Release blockers

A prompt is the only enforcement mechanism for permissions or allowed tools.
One long-lived credential grants broad access to several systems.
External content can be copied into privileged instructions without separation.
Generated code, commands, URLs, or SQL execute without validation.
Retries can repeat purchases, sends, deletes, or permission changes.
Operators cannot stop runs or identify what the agent changed.
There is no test set containing prompt injection and tool-failure cases.

Limitations

This checklist supports design review but does not replace threat modeling, secure code review, penetration testing, privacy assessment, or regulatory analysis. Risk depends on the data, users, integrations, jurisdiction, and consequences of error. Re-run the review after changing models, tools, permissions, retrieval sources, or execution environments.

Related planning

Confirm the process is stable with the automation readiness checklist. Choose the simplest architecture with RAG vs agents. Add spend and retry limits using the cost model, then evaluate tool stacks in the AI tools hub.