Bulkhead
Isolating resources (threads, connections, capacity) per dependency or tenant so a failure in one doesn't exhaust resources for others. Like watertight compartments in a ship. Prevents one slow dep from taking down the whole service.
- Multiple downstream dependencies where one can fail.
- Multi-tenant resource isolation.
- Single dependency (no isolation needed).
- When the overhead of pools isn't justified.
Definition
A bulkhead partitions resources (e.g. a separate thread pool per downstream) so a problem in one compartment can't consume all resources. If dep A is slow, only A's pool fills; B still works. Pairs with timeouts and circuit breakers.
Why it exists
Without isolation, one slow dependency exhausts shared threads/connections, breaking unrelated requests. Bulkheads contain failures to their source.
How it works
Allocate separate pools/limits per dependency or tenant. Calls to A use A's pool; if A hangs, A's pool fills and A's calls fail fast, but B's pool is untouched. Semaphore-based bulkheads limit concurrency without separate threads.
Scaling limits
Partitioning reduces utilization (idle capacity in one bulkhead can't help another). Too many fine-grained bulkheads waste resources. Tuning pool sizes per dependency is ongoing.
Cost shape
Lower utilization (reserved capacity per bulkhead). The cost is worth it to prevent cascading failures.
Alternatives
- Shared pool + circuit breaker — Better utilization but a slow dep still affects all until the breaker trips.
- Async non-blocking — No thread-per-call to exhaust, but still needs concurrency limits.
Failure modes
- Over-partitioning wasting resources.
- Pool sizes too small causing false rejections.
- Not isolating the right boundary.