What is Token usage?
Also called token consumption, token counts.
Token usage is the count of tokens a language model reads and writes for a request, normally reported as separate input and output totals. Tokens are subword units, so counts depend on the tokenizer as well as on text length. Usage is the main driver of both cost and latency in model-backed systems, and it is usually metered per request.
Providers report usage per request, separating tokens read from tokens written, because the two are metered differently. Some interfaces report additional categories, such as tokens served from a cached prompt prefix or tokens spent on internal reasoning before the visible answer. Counts come from the specific tokenizer in use, so the same sentence can produce different totals across models and across languages.
Usage drives two things at once. It is the primary determinant of what a request costs, and output length is the primary determinant of how long generation takes. It also interacts with the context window, since a conversation that accumulates history and tool results will eventually exceed the limit and must be trimmed or summarized, which is a behavior change and not only a budget one.
The classic surprise is that an agent loop resends the whole conversation on every step, so total usage grows much faster than the number of steps. Large tool results, retrieved documents, and pasted files often dominate input usage while the visible messages are tiny. Estimating tokens by counting words is unreliable, particularly for code, structured data, and languages that do not put spaces between words.
Token usage is the input to cost per action, and it is the first place to look when spend rises without traffic rising. It relates to latency through output length and to throughput through provider token limits, which are often enforced per minute alongside request limits. Reducing context, truncating tool output, and caching stable prompt prefixes are the standard levers.
Key points
- Reported separately for input and output, which are metered differently.
- Tokens are subword units, so counts vary by tokenizer, not just length.
- Agent loops resend history, so usage grows faster than step count.
- Large tool results can dominate input usage in a run.
In practice
An agent answers a question in four steps. Step one sends 900 input tokens. Each later step resends the growing conversation plus the previous tool output, so inputs reach 1,400, then 3,100, then 4,800 tokens, while outputs stay near 200 each. Total input usage is roughly 10,200 tokens for four short replies, which is why trimming tool output pays off more than shortening the prompt.