What is Chain of Thought Prompting?
Also called CoT, step-by-step prompting.
Chain of thought prompting asks a model to work through intermediate steps before giving a final answer, rather than answering immediately. Producing those steps improves accuracy on arithmetic, logic, and multi-step tasks, because each generated step becomes context for the next. The gain comes from the extra computation the steps allow, not from the model narrating a real internal process.
A model generating a single token has a fixed amount of computation available to it. Writing out intermediate steps effectively buys more, since every step is appended to the sequence and conditions everything that follows. This is why simply adding an instruction to reason step by step measurably improves results on problems that involve several dependent stages of work.
The written chain is not a transcript of the model's internals. Research shows that models can produce a plausible chain and then give a final answer inconsistent with it, or reach a correct answer through steps containing errors. Treating the visible reasoning as a faithful explanation, particularly when auditing a consequential decision, is a well documented mistake.
The costs are real and easy to underestimate. Chains consume output tokens, which are metered and produced sequentially, so responses take longer and cost more per request. On simple lookups the technique adds latency for no benefit, and on some short classification tasks it can degrade results by talking the model out of a correct first instinct.
Reasoning-focused models have absorbed the idea into training itself. They generate extended internal reasoning automatically, exposed either as a summary or hidden entirely, and are controlled by a reasoning effort setting rather than by a prompt instruction. Explicitly asking such a model to think step by step is usually redundant and occasionally counterproductive.
Key points
- Intermediate steps give the model more computation before answering.
- Helps on arithmetic, logic, and multi-step problems.
- The visible chain is not a faithful record of internal processing.
- Adds output tokens, latency, and cost; wasteful on simple lookups.
- Reasoning models do this automatically via an effort setting.
In practice
Ask directly: 'A shop sells pens in packs of twelve. If eight packs arrive and nineteen pens are sold, how many remain?' A snap answer is often wrong. Add 'Work through it step by step.' The model writes that eight times twelve equals ninety-six, then ninety-six minus nineteen equals seventy-seven, and answers correctly, because each computed step was available to the next.