💬
✉️
📸
▶️

Loading…

Design Instagram: media upload, CDN, and feed ranking

InstagramGuided Mode

Pre-Signed URLs: How Instagram Handles 100M Uploads/Day

Even with S3, Instagram's API was still proxying every upload — consuming massive CPU just streaming bytes. The Media Upload Service changed this: issue a pre-signed URL, let the client PUT directly to S3. API CPU for uploads dropped to near zero.

Request Flow

1

Client requests upload URL

The Poster calls the API for permission only — a lightweight request. No bytes transferred yet.

POST /api/photos/upload-url
→ { uploadUrl, photoId }
2

Upload Service signs the URL

The Media Upload Service generates a time-limited pre-signed URL cryptographically scoped to one S3 key. The client receives this URL.

const url = s3.getSignedUrl('putObject', {
  Bucket: 'photos',
  Key: `${userId}/${photoId}.jpg`,
  Expires: 300 // 5 minutes
})
3

Client uploads direct to S3

The Poster PUTs the image bytes straight to S3 / Blob Storage using the pre-signed URL. The API fleet is completely bypassed for the heavy data transfer.

fetch(uploadUrl, {
  method: 'PUT',
  body: imageFile,
  headers: { 'Content-Type': 'image/jpeg' }
})
4

Client confirms + API saves URL

After the upload completes, the client notifies the API which records the final S3 URL in the Database.

Why We Need This

With proxy uploads, every MB of image data consumed API CPU time for read/write. At Instagram scale (100M uploads/day), this would require a 10–30× larger API fleet solely for data piping.

Pre-signed URLs shift the bandwidth cost off your servers entirely. S3 handles the TLS termination and throughput. Your API handles only lightweight JSON requests.

Key Insight

Pre-signed URLs are the 'valet parking ticket' pattern: the client gets a time-limited, scoped credential to act on a resource directly. Your API issues the ticket but never touches the car.

Step 3 of 7