# What is Streaming Response?

Also called Response Streaming, Incremental Response.

A streaming response is an API reply delivered as a sequence of partial chunks while it is still being produced, rather than as one complete payload sent at the end. The client processes each chunk on arrival, which lowers perceived latency. Language model APIs commonly stream generated text token by token, alongside events describing tool calls and completion.

Mechanically, streaming relies on a transport that permits an open response body: chunked transfer encoding in HTTP/1.1, data frames in HTTP/2, Server-Sent Events, or a WebSocket. The server flushes each piece as it becomes available instead of buffering. The client must read incrementally, which means treating the body as a stream rather than waiting for a single parsed object.

The benefit is chiefly in time to first byte and perceived responsiveness. Total generation time is unchanged, but a reader can begin consuming output almost immediately, and a downstream system can start work on early results. For long outputs the difference between waiting for everything and seeing the first words quickly is substantial.

Streaming complicates error handling and validation. A failure that occurs midway arrives after a successful status code has already been sent, so protocols usually carry an explicit error event inside the stream. Structured outputs cannot be validated against a schema until the final chunk, which is why some clients buffer the full result before acting on it while still displaying partial text.

Cost accounting, content filtering, and retries all need care. Usage totals typically arrive in a terminal event rather than a header, moderation must operate on a growing buffer, and a retry after a partial stream may duplicate content unless the application tracks how much it already consumed.

## Key points

- Reply arrives as chunks while still being generated
- Improves time to first byte, not total duration
- Errors surface inside the stream after a success status
- Schema validation waits for the final chunk
- Usage totals usually arrive in a terminal event

## In practice

An assistant endpoint is called with streaming enabled. The client receives a start event, then dozens of small text delta events that it appends to the visible answer, then an event announcing a tool call with its arguments, and finally a completion event carrying token counts. The user sees text within a fraction of a second, while the application waits for the final event before recording usage.

## Related terms

- [Server-Sent Events](/en/glossary/server-sent-events)
- [WebSocket](/en/glossary/websocket)
- [API](/en/glossary/api)
- [Structured Output](/en/glossary/structured-output)
- [Tool Calling](/en/glossary/tool-calling)

[Back to the AI Glossary](/en/glossary)