# What is Greedy Decoding?

Also called greedy search, argmax decoding.

Greedy decoding is a text generation strategy that selects the single highest-probability token at every step, with no randomness and no lookahead. It is deterministic, so the same input and model produce the same output every time. It is the simplest decoding method and is equivalent to sampling at a temperature of zero.

At each step the model produces a probability distribution over the vocabulary. Greedy decoding takes the maximum and appends it, then repeats. Because no random draw occurs, the entire output is a deterministic function of the input, the weights, and the numerical execution path. Minor hardware or batching differences can still shift results, so determinism is practical rather than absolute.

The method's weakness is myopia. Choosing the locally most probable token at every step does not produce the most probable sequence overall. A slightly lower-probability token early on can open a much better continuation, and greedy decoding can never find it. This shows up as repetition loops and bland, generic phrasing on open-ended tasks.

Where correctness matters more than variety, that trade is usually worth taking. Structured extraction, classification, code generation, and any task whose output feeds an automated consumer benefit from reproducibility. Reproducible output also makes regression testing and debugging tractable, since a changed result points to a changed input or model rather than to sampling noise.

The alternatives address different failure modes. Sampling with temperature and nucleus cutoffs adds controlled randomness for creative work. Beam search explores several partial sequences before committing, addressing the myopia directly at higher compute cost. Which to choose depends on whether the task has one right answer or many acceptable ones.

## Key points

- Always picks the highest-probability next token
- Deterministic in practice, so outputs are reproducible
- Locally optimal choices do not yield globally optimal sequences
- Prone to repetition and generic phrasing on open tasks
- Equivalent to sampling at temperature zero

## In practice

A pipeline extracts vendor name, invoice number, and total from scanned documents into a fixed schema. Any variation between runs on the same document would look like a data change to downstream systems and would break the test suite. The team sets temperature to zero so the model decodes greedily. For the marketing copy generator in the same codebase they do the opposite, since a dozen distinct headline drafts is the point.

## Related terms

- [Temperature](/en/glossary/temperature)
- [Top-p Sampling](/en/glossary/top-p-sampling)
- [Beam Search](/en/glossary/beam-search)
- [Inference](/en/glossary/inference)
- [Token](/en/glossary/token)

[Back to the AI Glossary](/en/glossary)