πŸ’¬
βœ‰οΈ
πŸ“Έ
▢️

Loading…

Design Google Search: crawling, indexing, and ranking

Google SearchGuided Mode

πŸ”

8.5 billion queries/day. Results in <200ms.

Crawl pipeline β†’ Index pipeline β†’ Query-serving pipeline

Google Search: Three Pipelines

Request Flow

1

The scale

Google processes ~8.5 billion queries per day. Every query must return ranked, personalised results in under 200ms including network round-trips.

2

Three decoupled pipelines

Google Search is three pipelines running at completely different timescales: (1) the Query Processor + Search Coordinator + Ranking Service query-serving pipeline (<200ms); (2) the web crawl + Inverted Index build pipeline (hours/days); and (3) the Autocomplete Service + Trending Engine real-time suggestion pipeline (<20ms).

3

Why decoupled?

The crawl pipeline can be completely down for hours without affecting the query-serving pipeline. This decoupling is achieved via Kafka and hot-swappable index segments.

Key Insight

Google Search is not one system β€” it's three pipelines running at vastly different timescales, decoupled by design so each can fail, scale, and deploy independently.

Overview