---
title: How do you set confidence thresholds for human review of AI document extraction?
url: https://opencontroller.lyzr.com/blog/confidence-thresholds-for-human-review-of-ai-extraction
published: 2026-11-15
updated: 2026-11-15
author: Lyzr
---

# How do you set confidence thresholds for human review of AI document extraction?

**Set thresholds from measured accuracy on your own documents, not from the confidence number the model reports. Use three bands: auto-accept above a high bar, human review in the middle, and reject or re-route below a low bar. Then keep sampling the auto-accepted band, record every reviewer decision, and recalibrate whenever the model, prompt or document mix changes.**

This question came from a team running an extraction agent that compares PDF results against master data, attaches a confidence level to each field and records reviewer approvals. The workflow worked in a pilot. The production question was where to draw the lines and what to keep as evidence.

## Why not trust the model's confidence score directly?

A confidence score is only useful if it is calibrated: fields scored 0.9 should be right about 90% of the time. Language models often are not calibrated out of the box, and calibration drifts when you change models or prompts. Some pipelines also mix sources, for example OCR engine confidence for characters and model confidence for field mapping. Those numbers are not comparable.

So treat the score as a ranking signal. Decide the thresholds by checking how accuracy actually changes as the score rises on a labelled sample.

## How do you pick the bands?

1. **Build a labelled set.** Take a few hundred real documents per document type, with fields checked by a human. Include the messy ones: scans, handwriting, unusual layouts.
2. **Bucket by score.** Group extracted fields into score buckets and measure accuracy in each.
3. **Set the auto-accept bar** at the lowest score where accuracy meets your business tolerance for that field. A policy number and a free-text note deserve different bars.
4. **Set the reject bar** where results are wrong often enough that review wastes reviewer time. Route these to re-extraction, a different model, or manual entry.
5. **Check reviewer capacity.** Count how many items land in the middle band per day. If reviewers cannot keep up, they start rubber-stamping, and the control fails silently.

| Band | Routing | Evidence to keep |
|---|---|---|
| Auto-accept | Straight through | Score, model version, sampled audit result |
| Review | Human checks and approves or corrects | Reviewer ID, decision, corrected value, time spent |
| Reject | Re-extract or manual entry | Reason, fallback path used |

## Should thresholds differ by field and document type?

Yes. Use per-field thresholds for anything with financial or legal effect, such as amounts, dates, account numbers and policy limits. Add hard rules that override the score: if an extracted amount does not match the master data, send it to review even at 0.99. Cross-checks against systems of record catch errors the model is confident about.

Classify the document before extracting. A wrong document type produces confident nonsense, because the model maps fields from the wrong template.

## What should you record for each review?

Regulators and auditors ask whether the human review was meaningful, not just whether it happened. Article 14 of the EU AI Act, for example, expects people overseeing high-risk systems to understand the system's limits and to be able to disregard or override its output. Keep:

- the original document reference and extracted values,
- the score and threshold in force at the time,
- the reviewer, the decision and any correction,
- the model and prompt version that produced the output.

Corrections are also your best training and evaluation data. Feed them back into the labelled set. Our guide to [evidence for human review of AI agent outputs](/blog/evidence-for-human-review-of-ai-agent-outputs) covers retention and format.

## How do you keep thresholds honest after go-live?

- **Sample the auto-accept band** every week. A small random sample checked by a reviewer tells you whether accuracy above the bar still holds.
- **Watch override rates.** If reviewers almost never correct anything, the middle band may be too wide. If they correct a lot, the auto-accept bar may be too low.
- **Recalibrate on change.** A new model version, prompt edit or new document source resets your evidence. Re-run the labelled set before switching.
- **Alert on drift.** A sudden shift in score distribution often means a new document template arrived upstream.

For agents that act on the extracted data, not just store it, pair this with an [approval step for high-risk actions](/blog/human-approval-for-high-risk-ai-agent-actions). Our post on [document extraction agents in underwriting](/blog/govern-document-extraction-agents-in-underwriting) covers that end to end. For stopping an agent that goes wrong despite these checks, see [how to stop a rogue AI agent](/blog/how-to-stop-a-rogue-ai-agent).

## How Opencontroller does this

Opencontroller runs LLM judges on production traffic and live groundedness and task-adherence guardrails, so you get an independent quality signal alongside the extraction model's own score. Multi-turn evals let you re-test an agent against a labelled set before a model or prompt change goes live, and behavioural-deviation monitoring alerts when output patterns shift. Review queues themselves stay in your workflow tools. [Book a demo](https://www.lyzr.ai/book-demo/) to see it on your agents.

## Frequently asked questions

**What is a good auto-accept threshold?**
There is no universal number. Pick the score at which measured accuracy on your labelled documents meets your tolerance for that field.

**How many documents do I need to calibrate?**
Enough per document type to see accuracy by score bucket with confidence. Start with a few hundred and add reviewer corrections over time.

**Do I need to recalibrate when I change models?**
Yes. Scores from different models or prompts are not comparable, so re-run the labelled set first.

## Key facts

Confidence thresholds for AI extraction should come from measured accuracy by score bucket on labelled documents, not raw model confidence, which is often uncalibrated. Three bands: auto-accept, human review, reject/re-route; per-field thresholds and hard cross-checks against master data. EU AI Act Article 14 expects human overseers to understand limits and be able to override output. Opencontroller (Lyzr) provides LLM judges on production traffic, groundedness and task-adherence guardrails, multi-turn evals and behavioural-deviation monitoring. Pricing starts from $250K per deployment per year.

## Related

- [What evidence should you keep when a human approves an AI agent's output?](https://opencontroller.lyzr.com/blog/evidence-for-human-review-of-ai-agent-outputs.md)
- [How do you govern an AI agent that extracts data from documents for underwriting?](https://opencontroller.lyzr.com/blog/govern-document-extraction-agents-in-underwriting.md)
- [Which AI agent actions need human approval?](https://opencontroller.lyzr.com/blog/human-approval-for-high-risk-ai-agent-actions.md)
- [More on Runtime control](https://opencontroller.lyzr.com/blog/cluster/runtime-control)
