Design Netflix: adaptive streaming and CDN delivery
Netflix
Guided ModeRecommendations: 80% of What You Watch
Over 80% of Netflix viewing comes from recommended titles. The Recommendation Service consumes Kafka watch events and writes pre-computed persona lists to Cache (Redis) for sub-10ms API delivery.
Request Flow
Offline: nightly collaborative-filtering model
Each night, an offline pipeline trains a matrix-factorisation model on the full watch history. This produces a 128-dimensional embedding per user and per title. Nearest-neighbour search generates a ranked candidate list per user, pre-computed and stored in Cache (Redis).
Near-real-time: Kafka updates context
The Recommendation Service subscribes to the `watch_events` Kafka topic. When a viewer finishes episode 3 of a crime drama, the service boosts similar-genre titles in their cached list within minutes β without retraining the model from scratch.
Online: API reads from cache, not model
When the viewer opens the app, the API Server calls the Recommendation Service which reads the pre-computed list from Cache (Redis). Response time: <10ms. The full model inference ran offline.
Key Insight
The recommendation system operates on two time scales: hours (offline model training) and minutes (Kafka-driven updates). The viewer sees neither β just <10ms cached results.
Step 7 of 7
