Human Review

The problem

Some decisions belong to a person

Releasing money. Changing where money goes. Signing for the company. Turning someone down. An agent can be right about all of them and still should not be the one to decide.

Today most of those approvals happen in email. A clerk forwards a PDF, a manager replies "approved," and the evidence lives in three inboxes. The reply cannot show what the manager actually looked at. It can be forwarded, forged, or lost.

LaunchAI treats review as a step type: placed on the map where everyone can see it, given a screen designed for the decision, and recorded down to what the reviewer saw and chose.

A signature is not the same as a review

On August 11, 2020, Citibank meant to send about $7.8 million in interest to Revlon's lenders and instead wired roughly $894 million, the full outstanding principal. Three people reviewed and approved the transaction. The entry lacked settings that would have kept the principal from being sent, and none of the three caught it. Lenders holding about $500 million refused to return their share, and the case ran until the Second Circuit ruled for Citibank in September 2022.

$7.8M: Meant to be sent (interest)
$894M: Actually wired (full principal)
3: Approvals. One screen that did not make the consequence plain.

How a review runs

The agent waits. The run doesn't forget.

Whether the decision comes in four minutes or the next morning, the run continues as if no time had passed, and the record shows exactly how much did.

The agent stops
It reaches the review step and holds its place in the chain.
The item is queued
It lands in the review queue, and the assigned reviewer is notified.
One screen opens
The agent's record on one side; the source on the other: the PDF, the scan, the portal screenshot.
Every field is laid out
A corrected value writes through to the shared table, so the agent, the reports, and the next review all see it.
The reviewer decides
A rejection requires a reason, and the reason becomes part of the record.
The agent resumes
On the chosen branch, from exactly where it stopped.

Four doors

Four ways an item reaches a reviewer

A placed review step is the obvious door. There are three more, and a well-built workflow uses all four.

Workflow map

Placed

Every item that reaches that review point on the map.

Example: Every vendor bank change, before the bank field is edited.
Where you set it: The workflow map, as its own box.

Rule engine

Rule-routed

A decision rule sends some items to review and lets others pass.

Example: An invoice whose total exceeds the PO by more than the tolerance in the table.
Where you set it: A rule and its table on the Dual-Path Rule Engine.

Model path

Confidence gate

A model reading falls below the step's threshold.

Example: A smudged account number on a faxed form.
Where you set it: The model-path step.

Exception policy

Exception

The agent cannot finish a step safely: an ambiguous target, a missing fact, a write whose result is unknown.

Example: The screen froze after Post, and nobody can yet say whether the receipt posted.
Where you set it: The workflow's exception policy, on Atomic workflows.

The reviewer workspace

Each part answers a question the reviewer would otherwise go looking for

1
Source The document, email, or portal screen as an image, not a paraphrase.The reviewer checks the original, not the agent's summary of it.
2
The agent's record Every field the agent extracted or prepared, laid out.The conclusion under review.
3
Where each value came from Each extracted value tied to the region of the source it was read from.A wrong value is found in seconds, not by rereading the page.
4
Why it stopped The rule that fired, the reading below threshold, or the exception.The question the reviewer is actually being asked.
5
What each choice does The review step's own description, written when the workflow is built.Approve edits the bank fields; reject sends the request to the fraud path. Nobody has to guess.
6
The controls Approve, reject with a reason, correct a value, supply a missing fact, take over.Every outcome the policy allows, and nothing else.

Portal · Review · INV-20418 · Acme Supply

4Why it stopped Rule fired: invoice total exceeds PO by 6.1% (tolerance 2%)

1

Source · invoice.pdf

Qty 21

Total $12,604.00

2

Agent record

Vendor: Acme Supply
Quantity: 21 3
PO total: $11,880.00
Invoice total: $12,604.00
Job no.: missing

5What each choice does Approve posts the invoice. Reject returns it to the vendor with your reason.

6

Four decisions

The four decisions a reviewer can make

A

Approve

Confirms the prepared action.

Continues on the approved branch.

R

Reject with a reason

Stops the action and says why.

Follows the rejection branch to its own endpoint.

C

Correct a value

Fixes one field, such as a misread quantity.

Continues with the corrected value, written to the shared table.

S

Supply a missing fact

Adds what the agent could not find, such as a job number left off an invoice.

Continues with the fact in place.

Speed

Built so a review takes seconds

A review that takes longer than the task is a bottleneck with extra steps. The workspace is built against that.

The agent has already read, matched, and calculated. The reviewer checks a conclusion, not a stack of paper.

One screen

Everything needed to decide is on one screen.

Search and switch

Fields are searchable and layouts switchable.

One key to next

The next review is one key away.

Batch, where allowed

Low-risk items can be approved in a batch, where policy allows.

Portal or phone

Reviews can be taken in the Portal or on a phone.

Routes and escalates

Assignment follows role, region, or amount, and a review that waits too long escalates on its own.

Placement

Where to put review steps

Anywhere, or nowhere. Three review steps in the workflow that pays vendors; none in the one that files a weekly status report.
The agent skips any review step that is not in the map. Approvals your policy requires must be placed there deliberately, and the workflow map shows every one.
Put them before the irreversible click: submit, post, pay, sign. Once a transaction is in the ERP, stopping the agent does not undo it.
Anything that moves money, signs for the company, changes master data such as bank details, or turns a person down deserves a review.

Placement map

A placement map for common back-office actions

How that framework translates into LaunchAI's four doors. Amounts are illustrations; yours live in a table.

A rule-routed door keeps its band in a table, not in the rule's sentence, so the controller can move it without touching the workflow.
A door that never opens, week after week, is either placed in the wrong spot or guarding the wrong thing.

Placement map
Action the agent would take	Can it be undone cleanly?	Pattern in LaunchAI
Convert approved recommendations into purchase orders	Partly; a sent PO reaches the supplier	Placed review before conversion, reviewed as one list
Raise a customer's credit limit	Yes, but exposure starts at once	Rule-routed review above the band in the approvals table; below it, no pause
Post an inventory adjustment	Only by another adjustment	Rule-routed review outside the count tolerance
Submit a pay application to an owner's portal		Placed review, every submission
Send an adverse action notice to an applicant		Placed review, every notice
Update a vendor's remit-to address	Not in practice, once a payment goes out	Placed review, every change
Read an invoice and fill a coding worksheet		Confidence gate only

Routing and escalation

Every review step names who gets the item

The assignment is a lookup like any other: a table the business owns.

Condition	Assigned to (illustration)
Amount up to $4,999.99	AP specialist for the entity
Amount $5,000.00 and above	AP lead
Property in the Southeast region	That region's property accountant
Vendor master change of any kind	Controller

You set how long an item may wait before it moves to the next person on the assignment path, and alerts go out by the channels you choose: the Portal, email, or chat. A typical ladder: the assigned reviewer first; after four business hours, the team lead; by the next morning, the controller. The agent holds its place the whole time.

Batch approval

Batch approval, where policy allows

A speed feature, and also the fastest route to a rubber stamp. Decide per review step whether to permit it.

Batch only items that stopped for the same reason. A batch that mixes reasons hides the one item that stopped for a different one.

Reasonable to batch

Items that reached review for the same reason, such as a variance just outside a small tolerance
Low-value items below a band in the approvals table
Reversible actions, such as a status update
A list the agent built for one decision, such as this week's replenishment recommendations

Keep item by item

Any change to bank details or remit-to addresses
Payments or releases above that band
Filings, submissions, and signatures
Anything that turns a person down

Segregation of duties

Who builds, who approves, who reviews

Accounting has long required that no single person both create and authorize a consequential transaction. NIST's SP 800-53 carries the same idea as control AC-5, separation of duties.

LaunchAI divides the work into separate rights: building workflows, approving them for production, editing rules and tables, reviewing items, running, viewing history, and managing credentials. The backend enforces each one. Three splits matter most for review:

The person who edits an approval band should not be the only person reviewing the items that band routes.
The person who builds a payment workflow should not be the person who approves it for production.
For changes that redirect money, the person who verifies should not be the only person who approves.

Sensitive actions, such as approving a payment workflow, can demand step-up authentication: a fresh sign-in or a second factor at that moment.

Handoff

Taking over mid-run, and handing back

Handoff runs both ways. The agent hands a person a decision with the context attached, then takes the work back the moment the decision is made.

A person can also step in mid-run, do part of the job by hand, and hand back: a portal asks for a one-time code only a person can supply, or a record needs a judgment the workflow never anticipated. The agent resumes from the screen as the person left it. No values are copied between windows, and nothing has to be explained twice.

Worked example

A vendor bank-account change, verified by call-back

Fraudsters target this workflow because one unverified change redirects the next payment run.

The FBI's IC3 counted $55.5 billion in exposed losses across 305,033 business email compromise incidents between October 2013 and December 2023, and advises verifying account-change requests through a second channel. The pressure has not eased.

Since June 19, 2026, Nacha's fraud monitoring rules require every non-consumer ACH originator to keep risk-based processes reasonably intended to identify entries initiated due to fraud, reviewed at least annually. A forged bank-change request fits Nacha's definition of false pretenses. A documented, recorded call-back is one part of such a process; it does not by itself make a company compliant.

$55.5B: Exposed BEC losses, Oct 2013 – Dec 2023305,033 incidents · FBI IC3
$3.05B: BEC losses reported in 202524,768 complaints, ~$123,000 each
76%: Organizations hit by attempted or actual payments fraud in 2025AFP 2026 survey

Manufacturers verify supplier banking changes and property managers verify vendor banking changes, often many times a day. Here is the chain:

Read the change request
Whether an email, a PDF form, or a portal notice: vendor, new bank, routing and account numbers.
Open the vendor record in the ERP
Capture the current bank details and the phone number on file. That number comes from the vendor master or the original contract, never from the request, because whoever sent the request controls every phone number and email address in it.
Rule: every bank change gets a call-back
No exception for amount, vendor tenure, or urgency. Because it is a rule, nobody can argue it away on a busy Friday.
Create a call-back task for a person
With the number on file, the request, and the prior account side by side.
Human review
The reviewer calls the number on file, logs who answered, when, and what they confirmed, then approves or rejects. The review sits after the call-back task, so the change reaches a reviewer only once the call is logged.
Edit, or route to fraud
On approval, the agent edits the bank fields, captures the screen before and after, and notifies accounts payable. On rejection, it records the reason and routes the request to whoever handles suspected fraud.

A Action D Decision H Human review

Step 5

What the reviewer sees at step 5

The request exactly as it arrived, including the sender's full address, not just the display name.
The vendor record as it stands: bank name and the last four digits of the current account.
The requested bank name and the last four digits of the new account.
The number to call and where it came from: the vendor master, the original contract, or the state license listing.
Fields for the call: who answered, their title, the time, and what they confirmed.
The two outcomes, stated plainly: approve edits the bank fields; reject sends the request to the fraud path.

Variations

Three variations teams add

A second reviewer

Insert a review step after step 5, assigned to the controller, so the person who made the call is not the only one who approved the change.

A first-payment hold

A rule flags the first payment to any account changed in the last 30 days, and a placed review confirms it before release. The window lives in a table.

An outbound-only call

The reviewer places the call; an inbound call “from the vendor” never counts. A cloned voice can answer a phone, but it cannot choose which number your reviewer dials.

The review record

What the review record keeps

Who looked, what they saw, what they changed, what they decided, when, and how long the item waited. The record is filed beside the agent's own screenshots in the audit trail, so an auditor sees the machine's work and the person's judgment in one place.

That record answers the three questions a control test asks. Each answer is a field, not a recollection.

Was the approval made by someone with the authority to make it?
Was it made before the transaction, not after?
Was the approver shown the evidence?

What to measure

The review record produces the numbers directly

Track them per review step, because each step guards something different. Override rate and queue budget are defined in our human-in-the-loop guide; these are the operating measures around them.

Review measures
Measure	How to compute it	What it tells you
Items per step per week	Count of items reaching each review step	Whether a step's volume fits the people assigned to it
Decision mix	Share approved, rejected, corrected, and supplied	Which review steps are doing real work
Wait by reviewer and hour	Median and slowest decile of wait time, grouped	Where escalation or a second reviewer is needed
Escalations fired	Items that moved up the ladder ÷ items	Whether the first assignee is overloaded or absent
Corrections by field	Corrected values, grouped by field name	Which reading or rule to fix next
Batch share	Items approved in batches ÷ items approved	Whether batching stays within policy
Takeovers	Mid-run takeovers per 100 runs	Where the workflow meets situations it was not built for

Connections

How human review connects to other components

Atomic workflows Review is one of the three step types, with approved and rejected branches that each end somewhere on the map.Dual-Path Rule Engine Rules route items to review, and low-confidence model readings are gated there.Studio “Send it to me,” spoken while building, becomes a review step.Audit trail The review record sits beside the agent's screenshots.Run console Pending reviews show in live status, and a waiting run shows as waiting on review.

Governance and security

Reviewing is its own permission

Reviewing is its own permission. A person can hold it without the right to build, edit rules, or manage credentials.
Screenshots stay on the workstation that ran the agent. The Portal holds signed references and serves the evidence to authorized reviewers on request.
Every decision carries a name and a time. Rejections carry reasons; corrections carry old and new values.

Limits

Limits, stated as tradeoffs

Review is not free, and we don't pretend it is.

Review adds wait

An item that pauses for a person finishes later than one that does not. The remedy is placement, assignment, and escalation, not removing the review from steps that move money.

It cannot make a person careful

It can make the evidence hard to skip and the decision impossible to deny afterward. Attention is still the reviewer's job.

Chat approval buttons may cost less

For a simple yes or no on a flow that already runs through APIs. LaunchAI can run that same approval, plus the screen work around it, with the evidence and the record in one place.

Every review is a person's time

The goal is fewer, better reviews: rules for the common cases, people for the ones that need judgment.

Questions

Does a reviewer need LaunchAI installed on their computer?

No. Reviews are handled in the Portal, the shared web surface, from a browser or a phone. Only the machines that run agents need the desktop application.

Is a review step the same as approving the workflow?

No. They are two different approvals. A review step is a decision inside a run: this change, this record, today. Approving a workflow version is a decision about the procedure itself, made once before it goes to production. An approved version is read-only, so the chain a reviewer works inside cannot shift under them.

What should a reviewer do when the source document itself looks forged?

Reject it with a reason, and let the rejection branch carry it to whoever handles suspected fraud. Do not correct values taken from a suspect document; a corrected field on a forged request is still a forged request. The record keeps the document as it arrived, which is what an investigator will ask for.

Does correcting a value in review change the agent's rules?

No. A correction changes the value for that record in the shared table. Rules change only when someone with the right to edit rules and tables changes them. A correction that repeats every week is the evidence that a rule should.

Can the person who built a workflow also review its items?

The permissions allow it, and your policy decides. For workflows that move money or change master data, keep them apart: the builder shapes what the agent does, so a second person should judge what it produced.

More of the product

No prompts. No babysitting. No dumb questions.

Get a demo with our specialist team.

Get demo