Skip to content
Devgo

Twitter/X feed

Fan-out-on-write vs fan-out-on-read for a timeline

Twitter's home timeline is the classic fan-out problem. For most users, new tweets are pushed (fan-out-on-write) into precomputed timeline caches for their followers. For huge accounts (celebrities), fan-out-on-write would overwhelm, so their timelines are assembled on read by merging.

Product constraints

  • Timelines must load in <100ms for hundreds of millions of users.
  • A tweet by a celebrity reaches millions of followers — fan-out on write is explosive.
  • Tweets are write-heavy and ordered; reads are per-user timeline scans.
  • is acceptable for a few seconds.

High-level diagram

Twitter/X feed architectureA block diagram of the Twitter/X feed system's main components and request flow.AuthorTweet storeshardedFan-out workersRedis timelinesper userReader timelineClient
Illustrative block diagram of Twitter/X feed. Not to scale; see the request path below for flow.

Request path

  1. Tweet. User posts; the tweet is stored and an event fans out to update followers' timelines.
  2. Fan-out. For normal users, the tweet is pushed to each follower's precomputed timeline ( list). For celebrities, this is skipped.
  3. Read timeline. Normal user reads their precomputed list. Celebrity follower's timeline is built on read by merging recent tweets from followed celebrities into the cached list.
  4. Render. Timeline returned with engagement counts; caching and read replicas absorb load.

Data model

Tweets stored in a sharded store keyed by tweet id. Timelines are lists per user (precomputed). The social graph (followers) in a sharded graph/store. Hybrid: fan-out-on-write for normal, fan-out-on-read for celebrities.

Storage / cache / queue choices

Storage

Sharded tweet store (/Manhattan historically); social graph in a dedicated store.

Cache

lists as precomputed timelines — the core of the design.

Queue

Internal queues to drive fan-out workers asynchronously.

Scaling levers

  • Precompute timelines (fan-out-on-write) so reads are O(1) cache lookups.
  • Hybrid fan-out: skip write fan-out for celebrities, merge on read.
  • Shard tweets and the social graph by user id.
  • Cache timelines in ; rebuild lazily on cache miss.

Copy vs skip (for a smaller team)

Copy
  • Precompute timelines in a cache for fast reads.
  • Use hybrid fan-out (write for normal, read for celebrities).
  • Shard by user id; keep the social graph separate.
  • Make reads O(1) by pushing work to write time.
Skip
  • Pure fan-out-on-write for celebrity accounts (explosive).
  • Building timelines on every read from scratch (too slow).
  • A single timeline store (shard it).

, , ,

Sources