SQS
AWS's managed message queue for decoupling producers and consumers with at-least-once delivery. Standard queues offer high throughput and at-least-once; FIFO queues offer ordering and exactly-once processing. No broker to run.
- Decoupling workers, burst absorption, serverless task processing.
- When you want zero queue ops.
- Event streaming with replay and many consumers (use Kafka).
- Complex routing (use RabbitMQ/SNS).
Definition
SQS is a fully-managed queue. Producers send messages; consumers poll and delete after processing. Visibility timeout hides a message from others while processed; not deleted in time → redelivered. DLQs capture failures.
Why it exists
Running message brokers is operational burden. SQS gives an elastic, pay-per-use queue with no capacity planning, ideal for serverless and bursty workloads.
How it works
Producer sends to a queue URL. Consumer polls (long polling to reduce empties), processes, and deletes. Visibility timeout sets how long a message is hidden. Redrive policy moves repeated failures to a DLQ. FIFO uses a dedup ID and group ID for ordering.
Scaling limits
Standard: at-least-once, no ordering, possible duplicates. FIFO: 300 msgs/s per queue (with batching more). No fan-out to many consumers of the same message (use SNS+SQS). Polling adds latency vs push.
Cost shape
Per-request + storage. Blow-ups: empty short-polling requests (use long polling), many tiny messages, and DLQ accumulation.
Alternatives
- Kafka — Durable log with replay, but you run/manage it.
- RabbitMQ — Rich routing and ack semantics, self-hosted.
Failure modes
- Duplicates in standard queues (make consumers idempotent).
- Visibility timeout too short causing double-processing.
- DLQ filling unmonitored.