Design Instagram: media upload, CDN, and feed ranking
Pre-Signed URLs: How Instagram Handles 100M Uploads/Day
Even with S3, Instagram's API was still proxying every upload — consuming massive CPU just streaming bytes. The Media Upload Service changed this: issue a pre-signed URL, let the client PUT directly to S3. API CPU for uploads dropped to near zero.
Request Flow
Client requests upload URL
The Poster calls the API for permission only — a lightweight request. No bytes transferred yet.
POST /api/photos/upload-url
→ { uploadUrl, photoId }Upload Service signs the URL
The Media Upload Service generates a time-limited pre-signed URL cryptographically scoped to one S3 key. The client receives this URL.
const url = s3.getSignedUrl('putObject', {
Bucket: 'photos',
Key: `${userId}/${photoId}.jpg`,
Expires: 300 // 5 minutes
})Client uploads direct to S3
The Poster PUTs the image bytes straight to S3 / Blob Storage using the pre-signed URL. The API fleet is completely bypassed for the heavy data transfer.
fetch(uploadUrl, {
method: 'PUT',
body: imageFile,
headers: { 'Content-Type': 'image/jpeg' }
})Why We Need This
With proxy uploads, every MB of image data consumed API CPU time for read/write. At Instagram scale (100M uploads/day), this would require a 10–30× larger API fleet solely for data piping.
Pre-signed URLs shift the bandwidth cost off your servers entirely. S3 handles the TLS termination and throughput. Your API handles only lightweight JSON requests.
Key Insight
Pre-signed URLs are the 'valet parking ticket' pattern: the client gets a time-limited, scoped credential to act on a resource directly. Your API issues the ticket but never touches the car.
Step 3 of 7
