Design Dropbox: file chunking, sync, and storage
Dropbox
Guided ModeMetadata Cache: O(1) File Listings
Every time a device opens Dropbox, it requests the file tree for the user. Hitting the Metadata DB on every request is expensive under load. A Redis-backed Cache (LRU/FIFO) makes repeated metadata reads O(1) and keeps the DB load flat.
Request Flow
API checks cache first
The API looks up the user's file metadata in Redis. Cache key = user_id + path. If found, returns instantly — no DB hit.
GET file_meta:user123:/Documents/report.pdf
→ HIT: { name, size, chunks:[...] }Cache miss → Metadata DB
On a miss the Metadata Service queries the DB, populates the cache with a short TTL (60–300 seconds), then returns the result. Future requests are served from cache.
Invalidate on file change
When Device A uploads a new version, the Metadata Service invalidates the affected Redis key as part of write handling. The next read misses, re-fetches from the Metadata DB, and re-populates the cache. To keep the simulator graph acyclic, that invalidation is described here instead of modeled as a reverse edge.
DEL file_meta:user123:/Documents/report.pdf
Why We Need This
At Dropbox's scale, every device syncs its file tree on every reconnect. Without caching, those queries hit the Metadata DB — millions of reads per second. Redis handles 1M+ reads/second per instance at <1ms latency.
Key Insight
Cache invalidation on write + short TTL = strong consistency without hammering the DB. The trade-off: a user might see a stale listing for up to the TTL duration — acceptable for file metadata.
Step 8 of 8
