# What is Autoscaling?

Also called auto-scaling, elastic scaling.

Autoscaling is the automatic adjustment of computing capacity in response to observed load or a schedule. Horizontal autoscaling adds or removes instances, while vertical autoscaling changes the resources allocated to an existing instance. It matches capacity to demand without manual intervention, but reacts only after a signal appears, so it cannot absorb a spike faster than new capacity can start.

A scaling policy needs a signal, thresholds, and cooldowns. Processor utilization is the traditional signal but is a poor fit for workloads that spend most of their time waiting on network calls, which describes most agent and model driven work. Queue depth, oldest message age, in flight request count, or concurrent runs usually track the real constraint far better for those systems.

Reaction time is the fundamental limit. Between the load arriving, the metric being scraped, the threshold being crossed, the instance starting, and the application becoming ready, several minutes can pass. Sharp spikes are therefore absorbed by queues, buffers, and headroom rather than by scaling. Scheduled scaling ahead of known events, and a minimum capacity floor above zero, compensate for the lag.

Scaling down is the riskier direction. Removing an instance mid work loses that work unless the application handles termination signals, stops accepting new items, and finishes or safely returns what it holds. Long running agent tasks make this acute, since an instance may need many minutes to drain. Aggressive scale down combined with a rising load also produces oscillation, which cooldown periods exist to damp.

Autoscaling has real limits. It cannot fix a bottleneck that is not in the scaled tier, and adding application instances that all contend for the same database connection pool makes matters worse, not better. Nor does it remove cost concerns: a runaway loop scales its own resource consumption, which is why maximum capacity ceilings and spend alerts belong beside every scaling policy.

## Key points

- Adjusts capacity automatically from load signals or a schedule.
- Queue depth often beats processor use for waiting workloads.
- Startup lag means spikes need buffers, not just scaling.
- Scale down must drain work gracefully to avoid losing it.
- Always set a maximum ceiling; scaling amplifies runaway cost.

## In practice

An agent worker pool scales on the number of queued tasks per worker rather than processor usage, since workers spend most of their time waiting on model and tool calls. When depth exceeds twenty per worker for two minutes, capacity increases, up to a hard ceiling of forty workers. Scale down waits five minutes and signals workers to finish in flight runs before exiting, so no task is interrupted.

## Related terms

- [Worker](/en/glossary/worker)
- [Queue](/en/glossary/queue)
- [Concurrency](/en/glossary/concurrency)
- [Throughput](/en/glossary/throughput)
- [Multi-Tenancy](/en/glossary/multi-tenancy)

[Back to the AI Glossary](/en/glossary)