πŸ’¬
βœ‰οΈ
πŸ“Έ
▢️

Loading…

Design YouTube: video upload, transcoding, and CDN

YoutubeGuided Mode

Metadata Service: Track Processing State

A dedicated Service tracks every video's lifecycle: upload initiated β†’ processing β†’ ready β†’ published. This allows the UI to show progress and the pipeline to update status without coupling to the main API. The synchronous request path here is metadata creation; later pipeline status updates happen asynchronously.

Request Flow

1

Upload records metadata

When an upload is confirmed, the API calls the Metadata Service to create a record: video_id, uploader_id, title, s3_key, status='processing', created_at.

2

Pipeline updates status

After each pipeline stage completes, the Video Processing Pipeline calls the Metadata Service to update status (e.g. 'transcoding' β†’ 'segmenting' β†’ 'ready'). In production this pipeline runs as a worker fleet, so aggregate capacity is much higher than a single worker.

3

DB as source of truth

The Metadata DB stores: video_id, title, duration, s3_key, manifest_url, thumbnail_url, status, resolutions[]. This is what the API queries when returning video info to viewers.

Why We Need This

Without a Metadata Service, the main API DB grows to track all video state β€” mixing transactional data (user accounts) with high-churn video processing state. Separation keeps each database focused and scalable.

Key Insight

The Metadata Service is the single source of truth for 'is this video watchable yet?' β€” the pipeline writes to it, the streaming path reads from it. This decouples upload infrastructure from playback infrastructure.

Step 6 of 9