πŸ’¬
βœ‰οΈ
πŸ“Έ
▢️

Loading…

Design Twitter: feed fanout and timeline system design

TwitterGuided Mode

Async Fanout via Queue

Writing to 500 caches synchronously inside the API call would block the response. A Queue decouples posting from fanout β€” API returns immediately, fanout happens in the background.

Request Flow

1

API enqueues fanout job

After writing to the Database, the API pushes a fanout event to the Queue and returns 200 OK immediately.

POST /tweet
β†’ INSERT INTO tweets
β†’ queue.push({ tweetId, authorId })  // async
β†’ 200 OK  // returns in ~15ms, not ~500ms
2

Feed Service consumes queue

The Feed Service polls the Queue, fetches the follower list, and fans out to each follower's Cache.

// Background worker:
const { tweetId, authorId } = queue.consume();
const followers = db.getFollowers(authorId);
await Promise.all(
  followers.map(uid =>
    redis.lpush(`timeline:${uid}`, tweetId)
  )
);

Why We Need This

Queue persistence means fanout survives Feed Service crashes β€” jobs replay from the queue. Horizontal workers can process multiple fanout jobs in parallel.

Key Insight

The Queue absorbs traffic spikes (a viral tweet hitting thousands of workers simultaneously) and decouples the write API from the fanout pipeline. It's the buffer that makes async fanout reliable.

Step 4 of 6