What is Chain of Thought?
Also called CoT, step-by-step reasoning.
Chain of thought is the practice of having a language model produce intermediate reasoning steps before its final answer, rather than jumping straight to a conclusion. It measurably improves accuracy on multi-step problems such as arithmetic, logic, and planning. The written steps are an output of the model, not a transcript of how it actually computed the answer.
Generating intermediate text gives the model more computation to spend on a hard problem and lets each step condition on the last. A question that would require one dense leap becomes a series of smaller inferences, each easier to get right. The effect was documented in prompting research and is strongest on problems with genuinely sequential structure.
Early work triggered the behavior by adding an instruction to think step by step or by showing worked examples in the prompt. Newer reasoning-oriented models are trained to do this on their own, often producing hidden reasoning before a visible answer, and providers expose a setting for how much of that effort to spend. The underlying idea is unchanged.
The most important caveat is that the visible chain is not a reliable explanation. Studies have shown models reaching a correct answer while stating reasoning that does not support it, and stating reasoning that omits the cue that actually drove the answer. Treat it as a useful working artifact and a debugging aid, not as evidence of what happened internally.
In agents, chain of thought is the thinking half of the loop. It is what the thought step of the ReAct pattern contains, and it is what planning and task decomposition are built on. It also has costs: reasoning tokens are billed and add latency, so systems usually reserve heavier reasoning for steps where the decision is actually hard.
Key points
- Intermediate steps improve accuracy on multi-step reasoning tasks.
- Prompted in older models, trained into reasoning-oriented ones.
- Stated reasoning is not a faithful record of internal computation.
- Reasoning tokens add both cost and latency.
In practice
Asked which of three shipping options arrives first given different cutoffs and transit times, a model answering directly may pick wrongly. Prompted to work through it, it writes out each option's cutoff, adds the transit days, notes that one falls on a weekend, and compares the three dates. The final choice is the same kind of answer, but the arithmetic is now visible and far more often correct.