Review Work With AI Evals
AI Evals is the built-in workspace system that checks selected employee output against a clear standard before it reaches the people who rely on it.
TL;DR
AI Evals is the workspace's AI evaluation system. Think of it as your Quality Officer. You choose the workspace switch, the tools and reply checks to review, and the standards that matter to you. It uses a paid AI review only for checks you enable, and you manage it in Settings -> Technical -> AI Evals.
What Kind of System Is This?
AI Evals is a built-in workspace system, not a manager you hire and not a second employee doing the task. It is an AI quality evaluation system: it compares an employee's proposed output with a written standard and returns a score and reason.
It makes quality visible and can ask the same employee to correct a low-scoring draft within a small, controlled limit. It does not replace your judgment or silently invent a standard for your business. Read Your Workspace Systems for the shared model.
What It Can Review
AI Evals has two different controls. They are separate so you do not accidentally pay to review more than you intend.
| Review type | When it runs | What you control |
|---|---|---|
| Tool-specific review | When a selected tool produces a real deliverable, such as an outbound email or a finished file. | The workspace switch and the individual tool switch. |
| Every response | Before an employee reply is saved, but only for each response check you turn on. | Built-in checks, your own criteria, and any custom check. |
Ordinary chat does not enter a tool-deliverable review just because someone sent a message. An every-response check is the deliberate option when you want the Officer to look at every employee reply.
How It Works
- You turn on AI Evals for the workspace and enable the specific review you want.
- An employee produces eligible work or a reply reaches an enabled response check.
- The Officer compares the request and the output with the chosen standard.
- You see the score, verdict, and reason in the review log and activity record.
- For supported pre-delivery work, a weak draft can go back to the employee for a bounded correction before it is sent.
The standard is visible and editable. Start from the default, then add the business details that only you know, such as required wording, a house format, or a specific quality bar.
What You Control
| Control | What it changes |
|---|---|
| Quality Evaluation switch | Enables or disables the workspace-wide review system. |
| Tool-specific reviews | Selects which eligible tools can have their deliverables reviewed. |
| Every response checks | Chooses the built-in or custom checks that run on every employee reply. |
| Standards and criteria | Defines what good output means for your workspace. |
| Review log | Lets you inspect the score and reason rather than trusting a hidden verdict. |
Quality Officer is available on eligible plans. If the control is locked, the page tells you the required plan instead of quietly running checks you cannot manage.
What It Costs
Each enabled review is an additional AI check beyond the employee's own work, so it uses credits. The page shows Review cost and the Officer cost history shows the actual review charges recorded for your workspace.
That cost exists for a reason: the Officer reads the output against your chosen standard instead of assuming that a completed task was good. Keep only the tool and response checks that give you a useful signal. A review that is disabled does not make a review call.
What It Never Does
- It does not grade a person's own task or a casual chat message through the tool-deliverable path.
- It does not turn on every possible review by itself.
- It does not make a hidden third correction attempt when a draft still fails after the allowed corrections.
- It does not turn a review outage into a failed customer task. Work can remain unscored rather than being held up by the reviewer.
Tips and Tricks
- Start with the outbound work that matters most, such as customer email or finished files.
- Add a response check only when you want every reply evaluated. It is more coverage and more AI review work.
- Edit the standard when your business needs something more specific than the default, then inspect the review log to see whether it produces useful feedback.
- Use System Health when you want one workspace-level view of repeated quality problems or missing coverage.
Frequently Asked Questions
Q: Is AI Evals another employee? A: No. It is a built-in AI review system. Your employees do the work, while AI Evals checks enabled output against a standard.
Q: Does it review every message automatically? A: No. Tool-specific reviews apply only to enabled eligible tools. Every-response checks are separate opt-ins.
Q: Can I choose what “good” means? A: Yes. You can use the platform standard, edit it for your workspace, and add a custom response check with your own criteria.
Q: Why do AI Evals reviews use credits? A: Each review is an extra AI evaluation of the request and output. Disabled reviews do not run or consume credits.
Q: Will it block all work if a review has a problem? A: No. Supported pre-delivery work can receive bounded correction feedback. If the reviewer itself is unavailable, work can stay unscored rather than failing because of the review.