The short answer
Put a person where a wrong action is expensive and hard to undo, and where the agent's evidence is weak. Nowhere else. Show the source next to the agent's result, offer approve, correct or reject, and record the decision.
Why "review everything" fails
Two hundred items a day, 196 of them fine. The reviewer's job quietly becomes clearing the queue, not reading it. The four bad items pass with a signature on them, which is worse than no review: the record now says someone checked.
Hospitals learned this with drug-interaction alerts. Too many warnings, and people click through all of them, critical ones included.
Place review by cost and reversibility
Ask two questions about every step that changes something. What does a mistake cost? Can it be undone cleanly?
Then one override: weak evidence always goes to a person, whatever the quadrant. A fuzzy vendor match, a smudged scan, a value far outside normal, a first deal with a new customer.
Pick an example step, then choose the square it belongs in.
Five placement rules
- A review after a payment goes out is an incident report.
- "Amount above limit" or "new vendor" changes how someone reads it.
- If volume exceeds what a person can read properly, write rules for the common cases.
- Whoever edits the agent's rules should not be the only one approving its output.
- Anything that moves money holds or escalates. It never proceeds because nobody answered.
Four kinds of human step
Approval is only one. Each kind needs its own screen.
| Kind | What the person does | Example |
|---|---|---|
| Approve | Yes or no on a prepared action | Releasing supplier payments |
| Correct | Fixes one value, run continues | A misread quantity on a packing slip |
| Supply | Adds a fact the agent can't find | A job number missing from an invoice |
| Take over | Does a step by hand, hands it back | Answering a multi-factor prompt |
Corrections are gold. Each one is a ready-made test case: the source, the wrong value, the right value. Add it so the same mistake gets caught by a rule next time.
One screen, everything needed
If confirming an item means opening three systems, it gets approved without confirming. Put this on one screen:
- The source document as an image
- Each extracted value next to where it came from
- The current record the agent wants to change
- Why the item was flagged
- The proposed action
- Approve, correct, or reject with a reason code
Is review working?
| Metric | What it tells you |
|---|---|
| Review share | Whether the queue fits its budget |
| Override rate | Near zero for months: review catches nothing, switch to sampling |
| Queue age | Whether reviews stall the work |
| Sampled error rate | Whether unreviewed items are actually clean |
| Reason mix | Which rule to write next |
In LaunchAI Studio
Human review is a step you drop anywhere on the workflow map. The run waits, the reviewer decides, and the agent continues on the chosen branch. Product →
Questions
What happens to a rejected item?
It leaves the automated path on a branch you defined: return it, hold it, or hand it to a person. The agent never retries it on its own.
Can review be sampled?
Yes, for cheap, reversible steps. Pick items at random and widen the sample if errors rise. Never sample an irreversible step.
How many review steps per workflow?
At least one per irreversible action, plus wherever evidence is weak. Every extra one should catch something.
Keep learning
- Stop Hallucinated Numbers Never let the model have the last word on a number. Read the guide
- Testing Before Production Test an agent like a new hire with write access, only more often. Read the guide
- AI Agent Audit Trail A good trail lets anyone reconstruct each step: what the agent saw, what it did, which rule or model output decided it, what changed in the system, who approved, and when. Read the guide
No prompts. No babysitting. No Integrations.