Design Netflix: adaptive streaming and CDN delivery
Netflix
Guided ModeWatch History: The Data Heartbeat
While viewers stream, every 30 seconds their clients send a progress tick: { user_id, title_id, position_seconds }. The Watch History Service records these to Cassandra and publishes them to the same Kafka bus used by the upload pipeline β creating a unified event stream.
Request Flow
Progress ticks β write-optimised Cassandra
The Watch History Service buffers progress events and flushes to Cassandra every 30 seconds. Cassandra's wide-column model stores (user_id, title_id) β latest_position_seconds as an O(1) point-read β perfect for resume-from-anywhere across any device.
Kafka fan-out β many consumers, one topic
Every progress tick is published to Kafka's `watch_events` topic. Multiple independent consumer groups read it without interfering: the analytics pipeline (computing completion rates), content ops (monitoring engagement), and the Recommendation Service (updating personalisation signals in near-real-time).
Same Kafka as the upload pipeline
Both the upload pipeline and the watch event stream share the same Kafka cluster, partitioned by topic. Upload jobs and watch events never interfere β but sharing infrastructure means a single ops team manages one fleet instead of two.
Key Insight
Kafka is Netflix's universal event bus. Upload completions and viewing events are both durable, replayable logs. Any downstream system subscribes independently β that decoupling is what allows each consumer to scale, fail, and deploy independently.
Step 6 of 7
