Human in the Loop AI Design: Review Scales Only When It Shows One Decision and Its Evidence

Why does human review become the bottleneck rather than the model?

Human review does not fail because reviewers are careless. It fails because the interface hands them everything the model produced and asks them to reconstruct the decision themselves, and that reconstruction time is what decides whether the workflow survives its own volume.

I have watched this exact failure twice in the last year. A credit memo workflow ran at forty items a day during the pilot, with two analysts reviewing in a shared inbox. Everyone was happy. At four hundred items a day the same two analysts were the constraint, the queue grew by a day and a half each week, and the conversation inside the company turned from "is the model good enough" to "do we need to hire four more analysts". Nobody had questioned the model. The model was fine. The review interface was a scrolling document with a chat box under it.

Human review only scales when the human reviews a decision and its evidence rather than a wall of model output, which makes the interface the hard part of the system and the model the easy part.

That sentence is the whole argument. Everything below is the mechanism.

What is the difference between reviewing output and reviewing a decision?

A reviewer's real job is to make a ruling, not to read. Output review makes them read first, and reading is the unbounded part: the longer the generated text, the longer it takes to judge, and the more of it the reviewer skips. Decision review inverts the order — the ruling is presented first, along with the specific evidence that produced it, and the text becomes something you can inspect rather than something you must absorb.

The last row is the commercial point. Output review has a cost curve tied to how much text the system generates, which is a number your model choice can inflate. Decision review ties cost to ambiguity, which is a property of the business, and that number you can measure and reduce.

How do you get the evidence into the same view as the decision?

The interface has to carry five things, and a missing one of them puts the reviewer back to re-deriving the answer.

• The proposed action in the system's own language, not in prose. "Approve invoice 4471 for EUR 12,880 against cost centre 2103" is reviewable in a glance. Three paragraphs explaining that the invoice appears consistent with the purchase order are not.
• The rule that produced it, quoted, with the source location. If the ruling came from clause 4.2 of the supplier framework, the reviewer should see clause 4.2, not a summary of it.
• Two to four evidence items, each pointing at an exact span in the source. Not paraphrases. A reviewer who cannot click through to the line in the delivery note is trusting a paraphrase, which is the same position they were in before the model existed.
• What changed since the last comparable case. A delta badge — quantity up 40 percent, new supplier, payment terms shortened — gives the reviewer's experience something to work on instead of asking them to notice a difference inside a wall of generated text.
• The counterfactual. I ask the model for the single input that would flip the decision, and I show it: "this becomes a rejection if the delivery note shows 42 units instead of 24." This is cheap to produce and it converts a confidence score, which reviewers ignore after a week, into a specific thing to check.

Underneath that view, the run has to be reconstructable. The reason I insist on this is operational rather than academic: when a reviewer asks "why did it say this", the answer needs to be one click into the audit trail that lets a reviewer replay the exact inputs, versions and cost behind any decision rather than a meeting with whoever built the prompt. That capability is designed in at build time; retrofitting it into a workflow already carrying production volume is where the money goes.

One more decision in this section, and it is an interface decision rather than a modelling one: the reviewer needs three exits, not one. Approve, edit the decision, or reject with a reason code chosen from a short list. A free-text rejection box means nobody rejects, and the reason codes are the most useful data the system will ever produce.

What does the arithmetic of review look like?

Review time is the number that decides the business case, and it is almost never in the pilot report. The per-item times below are mine — measured with a stopwatch on two client queues, a document queue and an invoice exception queue — not a published benchmark. Treat them as the shape of the problem and measure your own queue before you commit to anything.

A loaded reviewer costs roughly EUR 72,000 a year, which is about EUR 36 an hour at 2,000 productive hours. Five reviewers is a EUR 360,000 recurring line. One reviewer with a better interface is a EUR 72,000 line plus a build cost that lands once. That gap is the entire return on the interface work, and it appears in the budget before any model improvement does.

Two caveats I would put in front of a CFO. The saving assumes volume stays high and stable; at thirty items a day, none of this matters and you should not build it. And the interface work is not free — a real decision view with evidence linking is a few weeks of build, not a dashboard someone assembles on a Friday.

Which decisions should a human make at all?

Interface design makes the remaining review cheap. Tiering decides how much review remains, and it is where the larger saving sits.

The placement question is not "how confident is the model". It is whether the action can be undone, whether a customer or regulator ever sees it, and whether a written rule exists at all. An action with no written rule behind it does not belong in Tier 1 no matter what the confidence score says, because there is nothing for the reviewer to judge it against and nothing for an audit to check it against.

What makes a review loop quietly fail?

• Rubber-stamping. Track the rate at which a later reader or a downstream audit reverses an approval. If the approval rate sits above 99 percent and nothing is ever reversed, the review is decoration. Two hundred approvals a day with no disagreement is not a signal of quality, it is a signal of attention that has run out.
• Queue depth. Reviewers make worse rulings at 200 open items than at 40. Cap the queue, rotate people, and treat a growing queue as a defect in tiering rather than a reason to work harder.
• Evidence that rots. Links into source documents break when storage paths change. A reviewer who has clicked three dead links will stop clicking, and the decision view silently degrades into an output view with extra chrome.
• Reviewing the easy cases. The tempting sampling policy is to check high-confidence items because they are fast. Attention belongs on the borderline band, which is what the counterfactual line is for.

What should you ask before funding this?

Ask four questions: what does the reviewer see on screen, how many seconds does one item take at your expected volume, what share of items lands in the full-decision tier, and who reverses an approval after the fact. If the answer to the first is a document and a chat box, the system has an unbounded headcount line attached to it and the model choice will not change that. Budget the interface with the same seriousness as the model, because review arithmetic is what decides whether this becomes a cheap workflow or a new department. The honest version of human-in-the-loop is not a person checking the machine's work; it is a machine presenting one decision, its rule and its evidence, in a form a person can rule on in under a minute.