Load shedding
Deliberately dropping or degrading lower-priority work when load exceeds capacity, so critical requests still succeed. It's the active alternative to letting everything degrade together. Usually graded: shed cheap/old/batch work first.
- Public endpoints with unpredictable spikes.
- Mixed criticality traffic on shared capacity.
- When every request is equally critical and you should instead provision more.
- Batch systems where you can simply queue and catch up later.
Definition
A strategy that rejects or defers work above a capacity threshold, returning errors fast or serving degraded responses, rather than queueing until latency and memory collapse.
Why it exists
Under overload, trying to serve everything means nothing works (latency collapse, OOM). Shedding keeps the important fraction healthy and responsive.
How it works
Classify requests by priority. At a concurrency or latency threshold, reject low-priority (503/429), drop batch jobs, or return cached/stale data. Adaptive shedding (like concurrency limits) tracks latency and backs off as it degrades.
Scaling limits
Choosing the threshold is the hard part — too low wastes capacity, too high causes the collapse you're avoiding. Needs good latency signals.
Cost shape
Minimal. The 'cost' is intentionally not serving some traffic, trading availability of the few for failure of all.
Alternatives
- Autoscaling — Adds capacity instead of dropping, but takes minutes and costs money.
- Backpressure — Slows rather than drops, preserving work but increasing latency.
Failure modes
- Shedding the wrong priority class.
- Shedding too late after latency already collapsed.