# What is Human Review Queue?

Also called review queue, human-in-the-loop queue.

A human review queue is a work list where AI outputs or pending actions wait for a person to approve, reject, or edit them before they take effect or reach a recipient. It converts an autonomous step into a supervised one for cases that are high risk, low confidence, or subject to policy requirements.

Queues differ by where they sit in the flow. A blocking queue holds the action until someone decides, which prevents bad outcomes but adds latency bounded by human availability. A non-blocking queue lets the action proceed and routes a copy for after-the-fact review, which preserves speed and catches patterns but cannot prevent the individual mistake. Choosing between them is a risk decision, not a technical one.

Routing rules determine what lands in the queue. Common triggers include an action class flagged as irreversible, a model confidence signal below a threshold, a monetary amount above a limit, a first interaction with a new counterparty, or a random sample used purely for quality measurement. Sending everything to review defeats the purpose, and sending too little means the queue provides false assurance.

The queue is only as good as the reviewer's context. A reviewer needs the original request, the reasoning or steps that led to the proposed action, the exact payload that will be sent, and a clear affordance to edit rather than only approve or reject. Reviewers who see hundreds of nearly identical items develop approval reflexes, so queue design that surfaces what is unusual about an item measurably improves catch rates.

Review decisions are valuable training and evaluation data. Every edit is a paired example of what the system produced and what a qualified person considered correct, which feeds directly into a golden dataset or a fine-tuning corpus. Capturing the edited version rather than a bare approve or reject flag is what makes the queue compound in value instead of merely absorbing labor.

## Key points

- Holds AI outputs or actions for human approval, edit, or rejection
- Blocking queues prevent mistakes, non-blocking queues preserve speed
- Routing triggers include risk class, low confidence, and sampling
- Reviewers need full context and an edit affordance, not just buttons
- Captured edits become high quality evaluation and training data

## In practice

An outbound messaging agent drafts replies automatically but any message to a contact marked as an enterprise account, or containing a commitment about pricing or delivery dates, is held. The reviewer sees the incoming thread, the draft, and the matched rule. Roughly 12 percent of drafts route to review, and about a third of those get edited before sending, with each edit stored as a paired example.

## Related terms

- [Online Evaluation](/en/glossary/online-evaluation)
- [Golden Dataset](/en/glossary/golden-dataset)
- [Kill Switch](/en/glossary/kill-switch)
- [Fail Open and Fail Closed](/en/glossary/fail-open-fail-closed)

[Back to the AI Glossary](/en/glossary)