What is Context Window?
Also called context length, context limit.
The context window is the maximum number of tokens a model can consider in a single request, counting the prompt, any attached documents, the conversation so far, and the response it generates. Anything outside that budget is simply not available to the model. Limits range from a few thousand tokens in older models to over a million in some current ones.
Everything the model knows about the immediate situation must fit inside this window. Applications that appear to have long memories are, underneath, re-sending relevant history on every request or retrieving passages from a store and pasting them in. The model itself carries nothing forward once a response finishes, so continuity is an application feature rather than a model feature.
When a conversation outgrows the window, something has to give. Common strategies are dropping the oldest turns, summarizing earlier history into a compact note, or retrieving only the passages relevant to the current question. Each of these loses information, so the design question is which losses are acceptable for the task rather than whether loss occurs at all.
A large window does not guarantee even attention across it. Measurements repeatedly show accuracy dipping for material buried in the middle of a very long input, a pattern often called the lost-in-the-middle effect. Filling a window to its limit can therefore reduce answer quality while also raising cost and latency, which surprises teams who treat capacity as free.
Input and output share the same budget in most implementations, so a very long document leaves less room for a long answer. Providers may also advertise a large window while separately capping output at a much smaller number of tokens. Checking both limits before designing around them avoids responses that stop mid-sentence for reasons that look mysterious.
Key points
- Counts prompt, attachments, history, and generated output together.
- Models retain nothing between requests; history is re-sent each time.
- Accuracy often drops for content buried mid-window in long inputs.
- Output limits are usually capped separately and are much smaller.
- Filling the window raises latency and metered cost.
In practice
Suppose a model has a two hundred thousand token window and you paste a hundred page contract that fills eighty thousand tokens. The system prompt takes another thousand and the running conversation takes five thousand. Roughly a hundred and fourteen thousand tokens remain for further questions and answers. Paste a second contract of similar length and the earliest turns must be dropped or summarized.