# What is Jailbreak?

Also called Model Jailbreak, Safety Bypass.

A jailbreak is a prompt or conversation designed to make a model produce content its safety training and usage policies are meant to refuse. Techniques generally work by reframing the request, for example as fiction, translation, research, or a hypothetical, so the harmful intent is less apparent to the model's learned refusal behavior.

Jailbreaking targets the model's own policy behavior, which distinguishes it from prompt injection. Injection is about whose instructions the model obeys. Jailbreaking is about getting a model to cross its own stated limits, and the person doing it is usually the operator of the session rather than a third party who planted content elsewhere.

The underlying weakness is that refusal behavior is learned from examples rather than enforced by a rule engine. Safety training generalizes imperfectly, so a request that is refused in one framing may be accepted in another that is semantically equivalent but distributionally unfamiliar. Long multi-turn conversations that shift context gradually are a persistent weak point.

Because a model cannot be relied on to police itself, defense is layered outside the model. Independent classifiers evaluate inputs and outputs without sharing the model's context. Content policies are enforced at the output stage rather than only requested in the system prompt. Rate limits, account-level monitoring, and anomaly detection catch systematic probing that a single-turn check would miss.

Jailbreak resistance is measured through adversarial evaluation, not asserted. Teams maintain suites of known bypass patterns, track the pass rate over time, and re-run them after every model or prompt change, since a change that improves helpfulness can quietly weaken refusal behavior. Publishing working bypass strings is generally avoided, as it hands a functioning exploit to anyone reading.

## Key points

- Targets the model's own refusal behavior, unlike prompt injection
- Works by reframing intent into an unfamiliar context
- Refusals are learned, so they generalize imperfectly
- Defense sits outside the model in independent classifiers
- Resistance is measured by repeated adversarial evaluation

## In practice

A team maintains a private suite of several hundred bypass attempts collected from public research and internal red teaming, each labeled with the policy it targets. The suite runs automatically whenever the system prompt or the underlying model changes. After one prompt revision aimed at making replies more direct, the bypass rate on a violence category rose measurably, and the change was reverted before release.

## Related terms

- [Prompt Injection](/en/glossary/prompt-injection)
- [Content Moderation](/en/glossary/content-moderation)
- [Red Teaming](/en/glossary/red-teaming)
- [Alignment](/en/glossary/alignment)
- [Output Safety](/en/glossary/output-safety)

[Back to the AI Glossary](/en/glossary)