Design YouTube: video upload, transcoding, and CDN
Youtube
Guided ModeMetadata Service: Track Processing State
A dedicated Service tracks every video's lifecycle: upload initiated β processing β ready β published. This allows the UI to show progress and the pipeline to update status without coupling to the main API. The synchronous request path here is metadata creation; later pipeline status updates happen asynchronously.
Request Flow
Upload records metadata
When an upload is confirmed, the API calls the Metadata Service to create a record: video_id, uploader_id, title, s3_key, status='processing', created_at.
Pipeline updates status
After each pipeline stage completes, the Video Processing Pipeline calls the Metadata Service to update status (e.g. 'transcoding' β 'segmenting' β 'ready'). In production this pipeline runs as a worker fleet, so aggregate capacity is much higher than a single worker.
DB as source of truth
The Metadata DB stores: video_id, title, duration, s3_key, manifest_url, thumbnail_url, status, resolutions[]. This is what the API queries when returning video info to viewers.
Why We Need This
Without a Metadata Service, the main API DB grows to track all video state β mixing transactional data (user accounts) with high-churn video processing state. Separation keeps each database focused and scalable.
Key Insight
The Metadata Service is the single source of truth for 'is this video watchable yet?' β the pipeline writes to it, the streaming path reads from it. This decouples upload infrastructure from playback infrastructure.
Step 6 of 9
