Load balancer
Distributes incoming traffic across multiple servers so no single one is overwhelmed and failed ones are skipped. It's the front door that makes a pool of machines look like one endpoint. Can be L4 (TCP) or L7 (HTTP).
- Any multi-instance service that needs one entry point and failure handling.
- Blue/green and canary deployments.
- Single-instance internal services where clients can talk to peers directly.
- When you need client-side load balancing with a service mesh instead.
Definition
A load balancer sits in front of a server pool and routes each request to a healthy member using an algorithm (round-robin, least-connections, hash). L4 balances TCP/UDP by connection; L7 inspects HTTP to route by path, host, or content.
Why it exists
Single servers can't handle all traffic or survive failure. A balancer spreads load and provides health checking, so the pool degrades gracefully as instances fail.
How it works
Health checks mark backends up/down. New requests go only to healthy ones. Session affinity (sticky sessions) pins a user to a backend via cookie/IP hash. L7 balancers can do TLS termination, retries, and content-based routing.
Scaling limits
The balancer itself can be a bottleneck or single point of failure, so production uses redundant balancers (active/active) and often DNS/anycast to spread across them. Connection tracking limits stateful L4 balancers.
Cost shape
Managed LBs charge per hour + LCU (processed bytes/connections). Blow-ups: huge connection churn, very high new-connection rates, or TLS handshake CPU at the LB.
Alternatives
- Client-side load balancing — No middle hop, but clients need the member list and health.
- Anycast — Network-level distribution to nearest node, not per-request balancing.
Failure modes
- Health check too lenient, sending traffic to a sick backend.
- Sticky sessions causing imbalance when one backend dies.