Design Netflix: adaptive streaming and CDN delivery
Netflix
Guided ModeUpload Pipeline: 300 GB β 1,200 Variants
Before any viewer can press Play, a title must pass through the upload and transcoding pipeline. Netflix uses pre-signed URLs so the studio tool uploads raw footage directly to S3 β the API never proxies a single byte of video data.
Request Flow
Pre-signed URL β API stays out of the data path
The Video Upload Service receives a POST /upload/initiate from the studio tool, generates a pre-signed S3 URL, and returns it immediately. The studio client streams the master file (up to 300 GB) directly to Blob Storage. The API's CPU during the upload: zero bytes handled.
S3 event β Kafka β Transcoding workers
When S3 confirms the upload, the Video Upload Service publishes an `upload.complete` message to Kafka. The worker pool begins parallel transcoding β multiple workers handle different time ranges of the same video simultaneously.
8 quality tiers, 2-second HLS segments
The Transcoding Pipeline produces H.264 and AV1 at 240p, 480p, 720p, 1080p, and 4K. Each resolution is split into 2-second segments and written back to Blob Storage. When all segments are ready, the title becomes streamable globally.
Why We Need This
Separating upload confirmation from transcoding completion means the studio gets a success response in under 1 second for a 300 GB file. Transcoding may take 4+ hours. The async Kafka pipeline allows independent scaling and per-stage retries without blocking the uploader.
Key Insight
Pre-signed URLs + async Kafka mean the upload API and transcoding workers are fully decoupled. You can re-transcode a title 100 times (codec experiments, HDR variants) without touching the upload flow.
Step 5 of 7
