AI Automation Readiness Checklist

Score the workflow before choosing tools

Give each dimension 0, 1, or 2 points: 0 means absent or unknown, 1 means partially defined, and 2 means documented and demonstrated with evidence. A maximum score is 16.

Dimension	Evidence for a score of 2
Process stability	Inputs, outputs, and transitions are documented and do not change weekly.
Acceptance criteria	A reviewer can distinguish an acceptable result from a plausible-looking failure.
Data access	Required data is available, permitted, current, and attributable to a source.
Exception handling	Common exceptions and an escalation owner are known.
Reversibility	Actions can be previewed, rolled back, or approved before commitment.
Evaluation set	Representative examples include normal, edge, adversarial, and failure cases.
Security boundary	Tools, credentials, users, and sensitive data have explicit access rules.
Economics	The value of an accepted outcome exceeds model, tool, review, and failure costs.

Interpret the total

13–16: ready for a bounded pilot. Start with one reversible path, a manual fallback, strict permissions, and a fixed evaluation set.
9–12: prepare first. Resolve the lowest dimensions before connecting production data or external actions.
5–8: redesign the process. Automation is likely to produce high review burden or unstable outcomes.
0–4: do not automate yet. Document the process and establish ownership before evaluating AI tools.

The total is not the only gate. A zero in security boundary, acceptance criteria, or reversibility blocks production use even when the total is otherwise high.

1. Process stability

Collect ten to twenty recent examples. Record which steps were actually performed, which systems were used, and where operators exercised judgment. If every case follows a different path, first standardize the process or narrow the automation to one repeatable subtask.

2. Acceptance criteria

Replace “good answer” with observable checks: required fields, supported citations, policy conditions, numerical tolerances, prohibited actions, or reviewer ratings. If two knowledgeable reviewers regularly disagree, document the disagreement and route uncertain cases rather than forcing automatic completion.

3. Data and permission access

List every source and account. Confirm ownership, retention, confidentiality, geographic, contractual, and user-permission constraints. The model should receive only the information required for the current step, not every document or credential available to the organization.

4. Exceptions and escalation

Sample failures before designing the happy path. Identify missing data, conflicting instructions, rate limits, stale records, unavailable services, and high-impact cases. Each exception needs a safe result: retry without side effects, ask for clarification, send to review, or stop.

5. Reversibility

Drafting and classification are easier pilots than sending, publishing, purchasing, deleting, or modifying permissions. Where an action cannot be reversed, require an explicit preview and human approval tied to the exact target and arguments.

6. Evaluation set

Create a versioned set of real and synthetic cases containing ordinary tasks, edge cases, adversarial inputs, and known failure modes. Store the expected outcome and review rule. Re-run it after model, prompt, tool, retrieval, or workflow changes.

7. Security boundary

Separate read, draft, and write permissions. Keep secrets out of prompts and logs. Treat web pages, email, documents, and retrieved content as untrusted data that cannot grant new authority to the system.

8. Economics

Calculate cost per accepted outcome, not cost per model request. Include retries, failed runs, external tools, infrastructure, monitoring, human review, and maintenance. Compare that total with the current manual process and the business value of faster completion.

Pilot contract

One named workflow owner.
One bounded task and one user group.
One versioned evaluation set.
One tool and data allowlist.
One stop control and manual fallback.
One budget, retry, and latency ceiling.
One weekly review of failures, corrections, and permission changes.

Evidence packet for the go/no-go review

Do not approve a pilot from the score alone. Attach the process map, twenty sampled cases, acceptance rubric, data inventory, permission matrix, exception list, rollback design, evaluation results, threat notes, and cost worksheet. For every score of two, link to the artifact that proves the capability. For every score of one, name the owner and deadline. A zero in a release-blocking dimension must have a decision to narrow or postpone the workflow.

Run the review with the workflow owner, an operator who handles exceptions, and the security or privacy owner when sensitive data or external actions are involved. Record assumptions such as expected volume, model version, tool availability, review staffing, and acceptable failure cost. The pilot should be paused when those assumptions stop being true rather than continuing under an obsolete readiness score.

Convert the score into remediation work

A low readiness score is a map of prerequisites, not an argument for a larger model. If inputs are inconsistent, create validation and ownership. If acceptance is subjective, collect examples and adjudicate disagreements. If exceptions dominate, simplify the process or route them explicitly. If external writes are not idempotent, repair the integration before adding retries or autonomy.

Assign every failed item an owner, evidence requirement, target date, and decision: fix, constrain, keep manual, or abandon. Re-score only after evidence changes. Updating a number because the team feels more confident weakens the gate.

Design a bounded pilot

Select one representative slice with reversible or reviewable outcomes. Freeze valid inputs, expected outputs, test set, data boundary, tools, identity, destinations, approval, step and spend limits, manual fallback, and operator. Use synthetic or test accounts where external effects are possible. Keep the current manual process available.

Run ordinary, edge, and failure cases before live traffic. Include missing data, ambiguous requests, unavailable tools, timeout, duplicate event, malformed result, policy denial, delayed approval, interrupted execution, and external state that changes mid-run. Validate the actual destination or business state rather than the model’s success message.

Declare pilot thresholds before results

Measure accepted outcome rate, critical error rate, intervention and correction, recovery without duplication, tail latency, total cost per accepted outcome, policy denials, trace completeness, and user or operator impact. Segment by case type and consequence. A strong average cannot compensate for one unauthorized payment, disclosure, message, or record deletion.

Set promote, continue, and stop rules. Promote only when all hard gates pass and the residual risk has an owner. Continue a pilot when failures are understood and safely contained. Stop when authority is broken, outcomes cannot be validated, rollback fails, or economics depend on hiding review and correction.

Review organizational readiness

Technical feasibility is only one dimension. Name the business owner for acceptance, operator for failures, security and privacy owners for access, and approver for scope changes. Confirm staffing for review and incidents, maintenance windows, support expectations, vendor ownership, and a budget for evaluation after launch.

Document who may change prompts, models, tools, policies, and credentials. Treat these as production changes with versioning, testing, approval, deployment, and rollback. An automation without an operating owner is not ready even when the prototype works.

Expansion record

When the pilot passes, expand one boundary at a time: volume, users, data sensitivity, tool access, destination, or action impact. Record evidence, changed risk, additional cases, owner approval, and rollback for each step. Re-run the readiness review when assumptions change rather than treating the initial score as permanent authorization.

Also record a decision not to automate. Manual work may remain correct when volume is low, exceptions require judgment, source data is unstable, or the consequence of error makes verification more expensive than execution. The checklist succeeds when it prevents a weak project as well as when it supports a strong one.

Review user impact before calling the pilot efficient. Measure delay, correction burden, ability to contest a result, accessibility, and whether automation transfers work to customers or downstream teams. Savings created by hidden rework are not accepted outcomes and should be charged back to the workflow.

Limitations

This checklist is an editorial planning tool. It does not establish compliance with privacy, employment, financial, medical, safety, or sector-specific requirements. High-impact decisions need qualified domain, legal, and security review.

Next steps

Choose between retrieval, workflow, and agent architecture with RAG vs agents. If the workflow uses tools, apply the security checklist. Estimate the pilot with the cost model before selecting a framework from the AI tools research hub.