A human review step catches errors only when the reviewer sees the evidence, faces a short queue and has the authority to overrule. Without those three, the step approves whatever arrives. The reviewer's name is on the record, but nobody is checking. Design the lane around what a tired person will actually do at the end of a long queue.
How a review step turns into a rubber stamp
The pattern is easy to predict. A reviewer gets a queue, nearly every item is correct, and the screen asks "approve or reject?" After a few days, the quickest answer is "approve". Reviewers are not careless. The design rewards speed and gives them no way to see why an item deserves a second look.
Three design choices cause most of it.
Sending everything to a human
Universal review feels safe. The queue grows until the AI path is no quicker than doing the work by hand, and attention spreads thinly over items that never needed it.
Route by confidence. Items the system is sure about go ahead. The human lane is for the items where the system is unsure, and the queue is short enough that each one gets real attention. In AI in loan origination, clean files reach the underwriter ready to approve, and files with a mismatch go to the exception lane.
Showing a verdict instead of evidence
"The system recommends approval" invites agreement. Disagreeing means rebuilding the reasoning from scratch, and the reviewer cannot see it.
Show the evidence and let the reviewer reach the conclusion:
- The value the system read, beside the page it came from.
- The passage a claim rests on, highlighted in the source.
- Each check that ran, with its result.
- The one thing that is uncertain.
Hiding where the doubt is
A single confidence figure tells a reviewer little. A sharp scan under an unclear policy and a clear policy on a blurred scan need different checks. Name the doubt in plain words, such as "salary read from a blurred photo", rather than "review required".
What regulators already say about oversight
The EU AI Act asks for oversight that works in practice. According to Article 14 of the Act, high-risk AI systems must be designed so that people can supervise them effectively, and the people overseeing them must stay aware of the tendency to rely too readily on the output. Source: EU AI Act, Article 14. The Act is not Indian law. It applies to you if you sell into Europe, and it states the design problem in the same terms as this article: oversight fails when a person trusts the output by habit.
What a good review screen contains
| Element | Why it is there |
|---|---|
| The named uncertainty | Points attention to the part that needs it |
| Source beside the extracted value | Turns checking into a glance |
| The failed check and what it expected | Shows the rule, not only the result |
| A one-step correction | A reviewer who must leave the screen to fix an error will approve instead |
| The decision recorded with a reason | Feeds the evaluation set and the audit log |
The last row pays back over time. Each correction is a labelled example, taken from the cases the system handled worst. Feed those into your evaluation set and the next release is tested on them.
The threshold is a business setting
The line between "proceed" and "send to a person" prices two mistakes: passing a bad item, and holding up a good one. Those costs differ by process and shift with conditions. A fraud screen needs a different line in a bad month than in a quiet one.
So the threshold sits in settings that the function bearing the consequence owns, and every change is logged. Set a first value from the evaluation set, where the trade-off is visible, then adjust against real outcomes. A threshold hard-coded by an engineer is a business decision made without the business.
Four numbers that show the lane is working
| Measure | Healthy reading | Warning reading |
|---|---|---|
| Override rate | Reviewers disagree with the system on a visible share of items | Close to zero, which suggests approvals by reflex |
| Time per item | Varies with the difficulty of the item | Drifts down to a few seconds for everything |
| Post-review error rate | Falls as the lane improves | Errors still pass with a human approval attached |
| Queue age | Stable, with old items cleared | Items waiting longer each week |
Read the first two together. A zero override rate with falling review time means the lane has stopped checking. Look for a cause in the design before you blame the people: too many items, no evidence on screen, or a threshold that sends the wrong items.
The third measure is the one that tells you whether the lane catches anything. Sample some approved items each month, check them independently and record the error rate.
Queue age matters because a growing queue forces reviewers to hurry. If it grows, either the threshold sends too much or the lane lacks staff. Fix one of those before the queue gets worse.
How to present it to a risk committee
A risk committee exists to hold judgement. Do not describe the system as one where software decides and a person signs off. Say this instead: the system prepares the file and flags what is uncertain, and a named person decides. Authority stays where it was. The preparation moves to software.
Bring the artefacts: the confidence threshold and who owns it, a sample review screen, the four measures with their readings, and the log that lets any decision be replayed. For the log, see the contents of an AI audit log. For the wider project risks, see why AI agent projects are cancelled.
Design checklist
- The routing rule is written down, with the threshold and its owner.
- Every item in the lane shows evidence, not only a verdict.
- The uncertainty is named in words.
- A reviewer can correct an item without leaving the screen.
- Every decision is stored with its reason and fed into the evaluation set.
- The four measures appear on a dashboard that someone reads each week.
- A named person can pause the automated path.
Questions buyers ask
How many items should a reviewer see in a day?
No fixed number works for every process. Watch time per item: when it falls to a few seconds, the queue is too long for the care each item needs. Cut the volume by tightening what the system sends, or add reviewers.
Should reviewers see the model's confidence figure?
Show it with the reason behind it. A bare figure invites trust or distrust without telling the reviewer where to look.
Who owns the threshold?
The business function that bears the cost of an error: credit, claims, operations. Engineering builds the control and the log, and the business sets the value.
Where Brihat Infotech fits
Brihat Infotech builds AI agents and automated workflows with confidence routing, exception lanes and a full action audit trail. The enterprise AI agents guide links the other pieces of the subject.