# What is Model Evaluation?

Also called Evals, AI Evaluation, Model Testing.

Model evaluation is the systematic measurement of a model or AI system against defined criteria using repeatable test sets. It spans capability benchmarks, task-specific accuracy on representative data, safety and refusal behavior, robustness to adversarial input, and regression testing after any change to a prompt, model, or retrieval configuration.

Public benchmarks and production evaluation answer different questions. A benchmark compares models on a standardized task and is useful for shortlisting. A production evaluation measures whether a specific system does a specific job on the data it actually sees, and a model that leads a benchmark can still underperform on a narrow domain task. Benchmark contamination, where test items appear in training data, further limits benchmark comparisons.

A usable evaluation set is built from real examples with agreed correct answers, covering ordinary cases, known hard cases, and past failures. Grading methods vary by task: exact match where there is one right answer, rubric-based scoring where quality is graded on stated criteria, and human review for judgment-heavy outputs. Using a model as a grader is efficient and carries its own biases, so it is usually calibrated against human labels.

Evaluation earns its cost through regression detection. Because system behavior can shift after a prompt edit, a retrieval change, a parameter adjustment, or a provider updating an underlying model, an evaluation suite that runs on every change is what turns a silent quality drop into a visible failing test. Without it, quality regressions are typically discovered by customers.

Safety evaluation is a distinct track with different economics. Suites cover refusal behavior across policy categories, resistance to known bypass patterns, personal data handling, and tool misuse. Where a capability evaluation reports an average score, a safety evaluation is often about the tail, since a rare failure on a consequential action can matter more than a small average improvement.

## Key points

- Benchmarks shortlist models, production evals measure your system
- Build sets from real examples, hard cases, and past failures
- Model graders are efficient and need calibration against humans
- Run on every prompt, retrieval, or model change to catch regressions
- Safety evaluation is about the tail, not the average

## In practice

A team maintains three hundred labeled support conversations spanning routine questions, ambiguous multi-issue tickets, and every case that previously produced a wrong answer. Each candidate change runs the full set, graded by rubric with a sample verified by a human. A prompt edit intended to shorten replies raised the average rating but dropped accuracy on multi-issue tickets, which the suite surfaced before release.

## Related terms

- [Red Teaming](/en/glossary/red-teaming)
- [Hallucination](/en/glossary/hallucination)
- [Model Drift](/en/glossary/model-drift)
- [Bias](/en/glossary/bias)
- [Alignment](/en/glossary/alignment)

[Back to the AI Glossary](/en/glossary)