What is Temperature?
Also called sampling temperature.
Temperature is a setting that controls how random a model's output is. At each step the model produces a probability for every possible next token, and temperature reshapes that distribution before one token is chosen. Low values concentrate probability on the most likely candidate, producing consistent output, while high values flatten it and admit less likely choices.
Mechanically, the model's raw scores are divided by the temperature value before being converted into probabilities. Dividing by a number below one exaggerates the gaps between candidates, so the leading choice dominates. Dividing by a number above one compresses those gaps, giving unlikely tokens a real chance of selection. A value of zero, where supported, means always take the highest scoring token.
Sensible ranges are task-dependent. Extraction, classification, code generation, and anything validated against a schema benefit from very low values. Brainstorming, naming, and creative drafting benefit from higher ones. Most APIs accept values from zero to one or two, but the useful band is narrower than the allowed one, and extreme values reliably produce incoherent text.
Temperature zero does not guarantee identical outputs across runs. Batching, floating point nondeterminism on accelerators, and provider-side model updates can all introduce variation. Lowering temperature also does not make a model more accurate. Randomness is reduced, so the model simply repeats its most confident answer, and that answer may be confidently wrong every single time.
Temperature and top-p both control randomness but through different mechanisms, and adjusting both at once makes results hard to reason about or reproduce. A widespread convention is to change one and leave the other at its default value. Some providers now restrict or ignore these settings on reasoning-focused models, where the sampling strategy is managed internally.
Key points
- Rescales token probabilities before one candidate is sampled.
- Low values give consistent output; high values give varied output.
- Zero reduces randomness but does not guarantee identical repeated runs.
- Lower temperature does not mean more factually accurate.
- Adjust temperature or top-p, but not both at once.
In practice
Ask a model to finish 'The best thing about Mondays is'. At a very low temperature it will almost always produce the same completion, perhaps something about a fresh start. Raise the temperature toward the top of the allowed range and repeated runs give a joke, a complaint, and an unexpected metaphor. The model's knowledge is unchanged; only the selection rule differs.