What is Tree Of Thoughts?
Also called ToT.
Tree of thoughts is a reasoning method in which a model generates several candidate intermediate steps at each stage, evaluates how promising each one is, and searches the resulting tree with strategies such as breadth first or depth first exploration, including backtracking. It generalizes single path step by step reasoning into a deliberate search over alternatives.
Standard step by step reasoning commits to one path. An early mistake propagates, because there is no mechanism to abandon a branch and try another. Tree of thoughts, introduced in 2023 research, makes the alternatives explicit: at each node the model proposes several possible next thoughts, a valuation step scores them, and a search procedure decides which to expand and which to prune.
The valuation step is the crux. It can be another model call asked to rate a partial solution's promise, a vote among sampled continuations, or an external check where one exists. Weak valuation makes the search expensive without making it better, since the procedure then explores arbitrary branches rather than promising ones.
Cost grows quickly. Branching factor multiplied by depth determines the number of model calls, so a modest tree can cost an order of magnitude more than a single reasoning pass. This confines the method to problems where correctness matters much more than latency and where a single pass demonstrably fails.
It suits tasks with checkable intermediate states and a real risk of early commitment, such as puzzles, constrained planning, and search style problems. It is a poor fit for open ended generation, where there is no meaningful way to score a partial answer and the tree becomes an expensive way to produce variation.
Key points
- Explores branching reasoning paths instead of one chain
- Requires a valuation step to score partial solutions
- Supports pruning and backtracking from bad branches
- Cost scales with branching factor times depth
- Fits checkable problems, not open ended writing
In practice
Given a scheduling puzzle with hard constraints, an agent proposes three possible first placements rather than one. Each is scored by checking remaining feasible slots, and the branch that leaves no valid slot for a required meeting is pruned immediately. Exploring the two survivors two levels deeper finds a valid schedule that a single greedy chain, committed to the pruned first placement, would have missed.