# What is Queue?

Also called message queue, task queue.

A queue is a buffer that holds units of work between the component that produces them and the components that process them. It decouples arrival rate from processing rate, so bursts are absorbed rather than dropped, and it allows work to be retried or redistributed if a processor fails. Queues are a foundational building block for background jobs and asynchronous agent execution.

The core property is decoupling in time. A producer can accept a request and return immediately once the work is durably enqueued, while consumers process it whenever capacity allows. This converts a hard capacity limit into a latency effect: during a spike, work waits in line instead of being rejected, provided the queue itself has room and consumers eventually catch up.

Delivery semantics are the detail that matters most in practice. Most production queues offer at least once delivery, meaning a message can be delivered more than once if an acknowledgment is lost. Exactly once delivery across a network is generally not achievable end to end, so consumers are usually designed to be idempotent instead. Ordering guarantees also vary, and strict global ordering typically costs parallelism.

Queue depth and age are the primary health signals. A steadily growing depth means arrival rate exceeds processing rate and the system is falling behind. The oldest message age indicates how stale the delayed work has become. Both are more useful than raw throughput, since a healthy queue can carry high throughput while an unhealthy one can look idle if producers have already given up.

Agent workloads suit queues well because individual runs are long and bursty. A queue lets a platform accept many scheduled or triggered agent tasks at once, meter how many run concurrently, isolate different classes of work into separate queues so one heavy workload does not starve another, and route repeatedly failing items to a dead letter queue for inspection.

## Key points

- Buffers work so producers and consumers scale independently.
- Absorbs bursts by converting overload into waiting time.
- At least once delivery is typical, so consumers need idempotency.
- Queue depth and oldest message age are the key health metrics.
- Separate queues isolate workload classes from each other.

## In practice

A platform schedules a nightly report for several hundred workspaces. Rather than running them all at once, each report is enqueued as a job and a pool of workers pulls from the queue at a fixed concurrency. Queue depth spikes to several hundred at midnight and drains over the following hour. If a worker crashes mid-job, the unacknowledged message returns to the queue and another worker retries it.

## Related terms

- [Worker](/en/glossary/worker)
- [Dead Letter Queue](/en/glossary/dead-letter-queue)
- [Throughput](/en/glossary/throughput)
- [Concurrency](/en/glossary/concurrency)
- [Idempotency](/en/glossary/idempotency)

[Back to the AI Glossary](/en/glossary)