Autoscaling
Automatically adjusting the number of instances to match load, scaling out under demand and in when idle. Reactive (metrics), predictive (scheduled/ML), or queue-based. Cuts cost and handles spikes, but has lag and thrash risks.
- Variable traffic where capacity can be added quickly.
- Stateless services.
- Stateful services that can't rebalance easily.
- When you need instant capacity (use a buffer instead).
Definition
Autoscaling adds or removes capacity based on signals (CPU, RPS, queue depth, latency). Horizontal scales instance count; vertical scales instance size. Policies set thresholds, cooldowns, and min/max bounds.
Why it exists
Traffic varies; fixed provisioning either wastes money at idle or fails under spikes. Autoscaling matches capacity to demand, improving cost and resilience.
How it works
A controller polls metrics and adjusts the desired count when thresholds are crossed, respecting cooldowns to avoid thrash. Predictive autoscaling pre-scales for known patterns. K8s HPA/VPA, cloud autoscaling groups, and KEDA (event-driven) implement it.
Scaling limits
Scaling reacts after the fact, so spikes can outpace scale-out (need headroom/buffer). Cooldowns prevent thrash but slow response. Downscaling can evict in-flight work. Stateful services are hard to autoscale.
Cost shape
Saves money at idle; can over-spend if mis-tuned (too aggressive). Blow-ups: thrash (scale up/down repeatedly) and over-provisioned max bounds.
Alternatives
- Fixed provisioning + buffer — Predictable and instant, but wastes idle capacity.
- Load shedding — Drops excess instead of scaling, lower cost, lost work.
Failure modes
- Thrash from too-sensitive thresholds.
- Scale-out lag letting latency spike.
- Downscale evicting in-flight requests.