beginnerintermediateadvanced

System Design · Section 88

Search System Design

Level: advanced
Read: 18 min
Concepts: 2

A search system is five stages that are usually thought of as one product, and separating them is what makes it operable. Ingestion moves documents from their sources into indexable form, with enrichment and filtering. Indexing turns those into the structures the engine queries — a write-heavy, batch-friendly workload sized by document volume rather than query rate. The query service turns typed text into a structured query, applying tokenisation, correction, synonym expansion and, critically, permission filters as part of the query rather than after ranking, since post-ranking filtering shrinks result pages and leaks document existence through counts. Ranking scores a bounded candidate set and is kept separate from matching because the two change at completely different speeds. And a result cache sits in front, unusually effective because search traffic is head-weighted — provided its key includes every input that changes the result set, permission scope above all. The five stages meet at exactly one place, the index, which is what lets each fail, scale and deploy independently: an ingestion outage degrades freshness rather than taking search down. Operating the index then comes down to six recurring facts, all consequences of the index being a derived, rebuildable copy. Lag between a change and its being searchable is inherent, because engines batch and refresh; the honest fix is a read-your-own-writes path served from the source rather than forcing a refresh per write, which trades a local correctness problem for a system-wide throughput one. Mappings are largely immutable, so a schema change means a new index: build it alongside, dual-feed both, verify, flip an alias, and keep the old one for a week so rollback stays one call. Shard count fixes the index's maximum parallelism for its whole life and cannot be changed without a reindex, so it is sized from projected volume. And hot terms are search's hot-partition problem, handled by caching, precomputation, or keeping the worst offenders out of the ranking path.

This section

Design a Basic Search Engine (Google or Bing) | System Design Interview PrepInterview Pen

What is true here

Five stages with independent scaling, failure and deploy profiles, meeting only at the index — which is what makes stage-level failure survivable.
Permission filters belong inside the query; filtering after ranking shrinks pages and leaks the existence of documents through result counts.
A result cache key must include permission scope, filters, locale, sort and index version — omitting scope makes the cache a cross-tenant leak.
Index lag is inherent; fix the visible case with a read-your-own-writes path from the source, not by forcing a refresh on every write.
Reindex by building alongside and flipping an alias; shard count is chosen once and cannot be changed without a full reindex.

What you will be able to do

Decompose a search system into five stages and state each one's scaling axis and failure signature
Build a result cache key that cannot leak documents across permission scopes
Run a zero-downtime reindex with an instant rollback path
Choose a shard count from projected growth, and recognise the symptoms of having chosen wrongly

The pipeline

Ingestion, indexing, query, ranking and caching as five components that meet only at the index.

The five stages: ingestion, indexing, query, ranking, caching

coreadvanced

A search system is five stages that people usually think of as one product, and separating them is what makes the whole thing operable. Ingestion is how documents get from wherever they live — a database, an event stream, a crawl — into a form the index can accept, which is also where enrichment, normalisation and filtering happen. Indexing turns those documents into the data structures a search engine actually queries, and it is a write-heavy, batch-friendly workload with completely different scaling behaviour from anything downstream. The query service parses what a user typed into a structured query: parsing, tokenising, spelling correction, synonym expansion, and applying filters and permissions. Ranking scores the matching documents, and it is deliberately separate from matching because the two change at different speeds — matching rules change rarely, ranking is tuned constantly, sometimes daily. Caching sits in front of the query path, where it is unusually valuable because search traffic is heavily head-weighted: a small set of popular queries accounts for a large share of volume, and their results change only when documents change. Splitting the five out matters because each has a different scaling axis, a different failure mode, and a different deploy cadence — you can reindex without touching the query service, tune ranking without reindexing, and lose the ingestion pipeline for an hour while search keeps serving slightly stale results.

Think of it as

A library, taken apart. Acquisitions receives new books and strips the packaging (ingestion). Cataloguing writes the index cards and files them (indexing). The enquiry desk turns "something about Roman roads in Britain" into a search of the card catalogue (query). The librarian decides which of the forty matching books to hand over first (ranking). And the shelf of most-requested titles by the door saves the whole round trip for the questions everyone asks (caching). Each of those has a different queue, a different staffing level and a different bad day, which is exactly why a library does not put one person in charge of all five.

python

def search(user, text, filters, page):
    key = cache_key(normalise(text), filters,
                    user.permission_scope,   # never
                    user.locale, page,       # omit
                    index_version())
    if hit := cache.get(key):
        return hit

    query  = build_query(text, filters,       # stage 3
                         user.permission_scope)
    hits   = engine.search(query, size=200)   # matching
    ranked = rank(hits, user)[:20]            # stage 4
    cache.set(key, ranked, ttl=60)
    return ranked

What we're doing: Follow one query through the read path, and one document update through the write path, and see where they meet.

search-pipeline-trace.txttext

WRITE PATH -- a product description is edited

  12:00:00  row updated in the product database
  12:00:01  change event published
  12:00:03  ingestion: fetch related data, build
            the document, drop internal fields
  12:00:04  indexing: bulk request queued
  12:00:06  index refresh -> the change is now
            visible to searches
            Total lag: 6 seconds.

READ PATH -- a user searches "waterproof jacket"

  cache key = ("waterproof jacket", filters={},
               scope=public, locale=en-GB,
               page=1, index_v=41)
  cache miss

  query service:
    tokenise, correct, expand synonyms
    ("waterproof" OR "water-resistant")
    add permission filter: visible_to:public
  index: 4,812 matches, top 200 returned
  ranking: score 200 candidates on relevance,
    availability, margin -> take 20
  cache set, ttl 60s

  Same query 900ms later: cache hit, 2ms.

WHERE THEY MEET: nowhere, directly. The write
path's only effect on the read path is that the
index and index_version change -- which is what
lets a six-second ingestion outage be invisible.

9: Six seconds is the index lag, and it is a design parameter rather than a bug. The next concept is entirely about what that number costs and how to keep the product honest about it.
21: The permission filter is part of the query, so the engine's match count of 4,812 already reflects what this user may see. Applying permissions after ranking instead would produce pages of fewer than twenty results and would leak, through the total count, how many documents exist that the user cannot read.
26: Ranking scores 200 candidates, not 4,812. Matching narrows cheaply using the index; ranking spends real computation on a bounded set — the same two-stage retrieve-then-rank shape a social feed uses.
33: The two paths are joined only by the index and its version. That decoupling is what makes stage-level failure survivable: ingestion can stop entirely and every query still returns correct, slightly older results.

Why this works: The trace shows why the five stages are worth naming: they meet in exactly one place, the index. That single join is what lets you reindex without redeploying the query service, tune ranking without touching ingestion, and survive an ingestion outage with degraded freshness instead of a search outage.

Omitting the permission scope from the cache key

Wrong

python

key = f"search:{query_text}:{page}"
# two users with different visibility share a
# cache entry; whoever misses first fills it

Better

python

key = f"search:{query_text}:{page}:{scope_hash}"
# scope_hash covers roles, team, tenant --
# anything that changes which documents match

What you see: A user occasionally sees documents they have no access to, in search results only, and never reproducibly — because it depends on which user filled the cache entry first. Opening any of the results returns a permission error, which is how it is usually reported.

Why: A cache key must include every input that can change the value, and permission scope changes the result set more than the query text does. Omitting it makes the cache a cross-tenant leak whose likelihood rises with cache hit rate — meaning the better the cache works, the worse the breach.

Two independent paths: documents in, queries through

Sources

database, event stream, crawl

Ingestion

enrich, normalise, filter

Indexing

write-heavy, batch-friendly

Index

sharded, replicated

Query service

parse, expand, filter by permission

Ranking

scores a bounded candidate set

Result cache

head-weighted traffic, keyed on every input

Results

Five stages, five different operational profiles

Five stages, five different operational profiles
Stage	Scales with	What breaking it looks like
Ingestion	Source change rate	New and updated documents stop appearing; existing search still works
Indexing	Document volume and update rate	Index lag grows; results go stale but stay correct
Query service	Query rate	Search is down — the only stage whose failure users see immediately
Ranking	Query rate × candidates scored	Results are still correct but ordered badly
Caching	Query rate and head-weighting	Load shifts onto the query service; latency rises

What must be in the cache key

What must be in the cache key
Input	Why it changes the result	Consequence of omitting it
Query text (normalised)	Obviously	Wrong results for a different query
Filters and facets	Different result set entirely	A filtered search returns unfiltered results
Permission scope	Different documents are visible	One user sees another user's documents
Locale / language	Different analysis and synonyms	Results in the wrong language or with wrong stemming
Sort and pagination cursor	Different slice of the same result set	Page 2 shows page 1
Index version	A reindex changes results	Stale results persist past the reindex

Remember: Five stages: ingestion (source to indexable document), indexing (write-heavy and batch-friendly), query (parse, expand, and apply permission filters as part of the query), ranking (score a bounded candidate set, tuned constantly), caching (head-weighted traffic, keyed on every input including permission scope and index version). They meet only at the index, which is what lets each fail, scale and deploy on its own — and lets an ingestion outage degrade freshness instead of taking search down.

Operating the index

Lag, reindexing, schema changes, shard sizing and hot terms — all consequences of the index being derived.

Index lag, reindexing, schema changes, shard sizing and hot terms

coreadvanced

Running a search system is mostly about six recurring operational facts. Index lag is the delay between a document changing and that change being searchable, and it is inherent rather than accidental: search engines batch writes and refresh periodically because refreshing on every write would destroy throughput. That makes search eventually consistent with its source, which is fine for search and misleading in the user interface — a user who edits something and immediately searches for it expects to find it, so the product usually reads the source directly for that case rather than pretending the lag is zero. Reindexing is unavoidable, because analysers, field types and mappings are largely fixed once an index exists, so a schema change means building a new index and switching to it. The safe pattern is to build the new index alongside the old, keep both fed by ingestion, verify the new one, then move an alias — which makes the switch atomic and the rollback instant. Shard sizing is a decision made once and painful to revisit: too few shards and a single shard becomes a bottleneck you cannot split without reindexing, too many and every query pays coordination overhead across shards that hold very little. Hot terms are the search equivalent of a hot partition — a small number of queries or values account for a disproportionate share of work, and they are handled with caching, precomputation, or by keeping the very expensive ones out of the ranking path. All six come back to the same idea: a search index is a derived, rebuildable copy, and the operations that matter are the ones that let you rebuild and switch it without an outage.

Think of it as

Treat the index as a build artifact, not a database. You would not edit a compiled binary in place; you would change the source and rebuild. A search index is the same: the source of truth lives elsewhere, the index is compiled from it, and the interesting operations are all about building a new one safely and switching over. Once you hold that view, reindexing stops being an emergency and becomes a routine deploy — which is exactly the property you want on the day a mapping turns out to be wrong.

bash

# reindex safely: build alongside, then flip
PUT   /products_v42            # new mapping
POST  /_reindex                # copy from v41
# ingestion writes to BOTH v41 and v42 meanwhile
GET   /products_v42/_count     # verify
POST  /_aliases                # atomic switch
  { "actions": [
      { "remove": { "index": "products_v41",
                    "alias": "products" }},
      { "add":    { "index": "products_v42",
                    "alias": "products" }} ]}
# rollback = the same call, reversed

What we're doing: Watch index lag become a support ticket, then fix it in the product rather than in the engine.

read-your-own-writes.txttext

Seller renames a product at 12:00:00.

12:00:00  database updated. The seller is
          redirected to their product list.
12:00:00  the product list is a SEARCH query.
          The index still holds the old name.
12:00:00  seller sees the OLD name and files a
          bug: "renaming does not work"
12:00:06  index refresh; the new name appears

The engine is behaving exactly as designed.
The product is not.

Fix, in order of preference:

1. Read the source for the actor's own recent
   changes -- the seller's own product list
   comes from the database, not the index.
2. Return the edited document from the write
   response and merge it into the next view,
   so the UI is correct before the index is.
3. Show freshness explicitly ("updated a few
   seconds ago, search may lag") where 1 and 2
   do not apply, such as a shared team view.

What NOT to do: force a refresh on every write.
It fixes this ticket and cuts indexing
throughput badly enough to create a much larger
lag problem under load.

5: The root cause is that a list which feels like "my data" is served by a system that is eventually consistent with that data. Nothing is broken; the freshness guarantee of the read path does not match the user's expectation of it.
17: Reading the source for the actor's own recent changes is the cheapest fix and the one that generalises: the user who made a change is the only one who reliably notices lag, and their view is usually narrow enough to serve from the database.
29: Forcing a refresh per write is the reflex fix and it trades a small, local correctness problem for a system-wide throughput problem — the classic shape of an optimisation that makes the metric you were watching better and everything else worse.

Why this works: Index lag is not a defect to eliminate but a property to design around, and the design work happens in the product rather than in the engine. Naming which views need read-your-own-writes, and serving exactly those from the source, keeps the engine tuned for throughput while removing the one case where lag is visible.

Reindexing in place by deleting and rebuilding the live index

Wrong

bash

DELETE /products
PUT    /products          # new mapping
POST   /_bulk             # reindex from source
# search is empty for the entire rebuild, and
# there is nothing to roll back to

Better

bash

PUT   /products_v42       # build alongside
POST  /_reindex           # while v41 still serves
POST  /_aliases           # atomic flip when ready
# rollback: flip the alias back

What you see: Search returns no results for the length of the rebuild — minutes to hours depending on corpus size — and if the new mapping turns out to be wrong, the only way back is another full rebuild with the old mapping.

Why: An index is a derived artifact, so there is no reason to have only one. Building the replacement alongside the original costs disk and keeps every property you want: the old index keeps serving, the new one can be verified against live traffic before it takes any, and the switch and its reversal are both a single metadata operation.

A reindex with no downtime and an instant rollback

Day 0
Create products_v42
new mapping; the alias still points at v41
Day 0
Dual-feed ingestion
every change is written to both indexes
Day 0–1
Backfill v42 from v41
historical documents copied in the background
Day 1
Verify
counts, spot-checked queries, ranking comparison on live traffic
Day 1
Flip the alias
atomic; no client changes and no deploy
Day 1+
Keep v41 for a week
rollback is one alias call until it is dropped

Six operational facts, and what each costs if ignored

Six operational facts, and what each costs if ignored
Fact	Handled by	Cost of ignoring it
Index lag	Publish the lag as a metric; read the source for read-your-own-writes	Users report "my edit did not save" for something that saved correctly
Reindexing	Build-alongside plus an alias flip	Schema changes require downtime, so they stop happening
Schema changes	Treat mappings as immutable; version the index name	A wrong field type is discovered in production and cannot be fixed in place
Shard sizing	Size from projected document count and growth, not current volume	A single hot shard caps throughput, fixable only by a full reindex
Hot terms	Cache popular queries; precompute; cap expensive ranking work	A handful of queries dominates latency for everyone
Eventual consistency	State the freshness guarantee in the product, not just the runbook	Every stale result is triaged as a bug

Shard count: the two failure directions

Shard count: the two failure directions
Property	Too few shards	Too many shards
Symptom	One shard is hot; adding nodes does not help	Every query fans out widely for little data each
Cost	Indexing and query throughput capped by one shard	Per-shard coordination and memory overhead dominate
Fixable without a reindex?	No	Sometimes, by shrinking
Rule of thumb	Size from projected documents and growth, not today's volume	Prefer fewer, larger shards until measurement says otherwise

Remember: The index is a derived, rebuildable artifact, and every operational answer follows from that. Index lag is inherent, so state the freshness guarantee and serve read-your-own-writes from the source instead of forcing refreshes. Schema changes mean a new index: build alongside, dual-feed, verify, flip an alias, keep the old one for a week. Size shards from projected volume, because a shard cannot be split later. And treat hot terms as hot partitions — cache, precompute, or keep the worst out of the ranking path.