When a user types "best tacos near me" into [Yelp](/learn/system-design/questions/yelp), they expect results ranked by relevance, not raw database rows. When someone searches for problems on [LeetCode](/learn/system-design/questions/leetcode), they expect partial matches, tag filters, and difficulty sorting all at once. Relational databases are great at exact matches and range filters, but they fall apart when users expect fuzzy matching, synonym handling, and relevance ranking. That is where **full-text search** comes in. The core idea: instead of scanning every row in a table to find matches, you build a specialized data structure — an **inverted index** — that lets you look up any word and instantly get back every document containing it. The rest of this pattern explains how that index works, how Elasticsearch distributes it across a cluster, how to score results so the best ones come first, and how to keep everything in sync with your primary database. ## The Inverted Index — Why Regular Databases Struggle Think about how a relational database stores data. Each row is a document, and to find all documents containing the word "caching," the database must scan every single row and check if the text column contains that word. That is a full table scan — O(n) — and it gets slower as your data grows. An **inverted index** flips the structure. Instead of mapping documents to their contents, it maps every unique term to the list of document IDs containing that term. Looking up "caching" is now a single hash-table lookup — O(1) — regardless of how many documents you have. This is exactly the same insight behind database indexes (B-Trees map column values to row IDs), but inverted indexes are designed specifically for text. They handle things B-Trees cannot: partial matches, stemming (matching "running" when searching for "run"), and multi-word queries across multiple fields. **See it in practice:** The [Scaling Reads](/learn/system-design/design-patterns/scaling-reads) pattern covers B-Tree indexes for structured queries. When your queries are keyword-based, fuzzy, or need relevance ranking, you need the inverted index approach described here. ## The Analysis Pipeline — From Raw Text to Searchable Tokens Raw text cannot go directly into an inverted index. The string "The Redis Caching Guide is GREAT!" needs to be broken down and normalized so that a search for "redis caching" actually matches it. This happens through a configurable **analysis pipeline**. The pipeline has four main steps: 1. **Tokenization** splits text into individual words. "Redis Caching Guide" becomes three separate tokens. 2. **Lowercasing** normalizes case so "Redis" and "redis" match the same index entry. 3. **Stop word removal** drops common words like "the," "is," and "at" that appear in almost every document and add noise without improving relevance. 4. **Stemming** reduces words to their root form. "running," "runs," and "ran" all become "run." This dramatically increases **recall** — the percentage of relevant documents your search finds. An optional fifth step, **synonym expansion**, maps equivalent terms so "NYC" matches "New York City." This is especially useful in e-commerce search where users describe the same product in many different ways. Each step is configurable per field. A title field might skip stemming to preserve exact brand names, while a body field applies the full pipeline for maximum recall. The trade-off is always the same: more analysis steps mean more matches found (higher recall) but potentially more irrelevant results (lower precision) and a larger index. ## Elasticsearch Architecture — Distributing the Index A single-node search engine works fine for small datasets, but when you have millions of documents and thousands of queries per second, you need to distribute the index across multiple machines. **Elasticsearch** is the most widely used solution for this. Elasticsearch organizes data using four core concepts: When you create an index, you choose how many **primary shards** to split it into. Each primary shard is placed on a different node, and each gets one or more **replica shards** on other nodes. Writes always go to the primary shard first, then replicate to replicas. Reads can be served by any copy — primary or replica — which means adding replicas linearly scales read throughput, just like [read replicas](/learn/system-design/design-patterns/scaling-reads) in a traditional database. The key architectural insight: each shard is a complete, independent Lucene inverted index. Elasticsearch coordinates across shards but delegates the actual indexing and searching to Lucene. This is why Elasticsearch scales — it is a distributed coordination layer on top of a proven single-node search engine. **See it in practice:** In [Yelp](/learn/system-design/questions/yelp), the restaurant index is sharded across nodes so geo-filtered searches return in milliseconds even with millions of listings. The [Scaling Writes](/learn/system-design/design-patterns/scaling-writes) pattern covers sharding strategies in depth — the same principles (choosing shard keys, handling hot shards, rebalancing) apply to search indexes. ## Search Query Flow — From Keywords to Ranked Results When a user submits a search query, it goes through several stages before results come back. Understanding this flow is essential for designing a system that returns relevant results quickly. The query flows through four stages: 1. **Analyze**: The query text goes through the same analysis pipeline as the indexed documents — tokenize, lowercase, remove stop words, stem. This ensures the query terms match the indexed terms. 2. **Lookup**: Each analyzed term is looked up in the inverted index to retrieve matching document IDs. For multi-term queries, Elasticsearch intersects or unions the posting lists depending on the query type (AND vs OR). 3. **Score and rank**: Matching documents are scored using **BM25** (Best Match 25), the default relevance algorithm. BM25 considers three factors: **term frequency** (how often the term appears in the document, with diminishing returns), **inverse document frequency** (rare terms are weighted higher than common ones), and **document length normalization** (shorter documents with the same term count score higher because the term is more concentrated). 4. **Return top-K**: Only the highest-scoring documents are returned to the user. In a sharded index, this becomes a **scatter-gather** pattern. The coordinating node sends the query to every shard in parallel. Each shard runs the full pipeline locally and returns its top-K results. The coordinating node then merges all shard results and performs a global re-ranking to produce the final top-K. ### Beyond BM25 — Vector Search BM25 matches on literal terms. But "affordable housing" should match "low-cost apartments" even though they share no words. **Vector search** solves this by converting both documents and queries into high-dimensional numerical vectors using a language model (an embedding model). Relevance is measured by **cosine similarity** — vectors pointing in similar directions represent semantically similar content. Modern search systems combine both approaches: BM25 for keyword precision and vector search for semantic recall, merging results using **reciprocal rank fusion**. This gives you the best of both worlds — exact keyword matches when users know what they want, and semantic matches when they describe it differently. ## Multi-Faceted Search — The Yelp Pattern In real applications, search queries combine multiple dimensions in a single request. The [Yelp](/learn/system-design/questions/yelp) question is the canonical example: - **Full-text search**: "best tacos" matches against restaurant names and review text using BM25 - **Geospatial filter**: "within 5 miles" uses a geo_distance query on lat/lng coordinates - **Category filter**: "Mexican restaurants" is an exact-match term filter on a category field - **Rating sort**: "highest rated first" sorts by a numeric avg_rating field Elasticsearch handles all of these in a single query using its bool query structure. **Filters** (geo, category) are cached and do not affect relevance scoring — they simply include or exclude documents. **Match clauses** (full-text search) contribute to the BM25 relevance score. **Sort clauses** can override or supplement relevance-based ordering. This composability is one of Elasticsearch's biggest strengths. You can mix full-text, geo, numeric, and exact-match queries in any combination without writing different query paths for each. ### Why Filters Execute First Elasticsearch optimizes multi-faceted queries by executing **filters before scoring**. Filters (geo, category, price range) are simple yes/no checks that narrow the candidate set cheaply. They are also cached in a bitset — a compact data structure where each bit represents whether a document passes the filter. On subsequent queries with the same filter, Elasticsearch skips re-evaluation entirely and reuses the cached bitset. Only after filtering does the engine run the expensive BM25 scoring on the surviving documents. This means a search for "tacos" within 2 miles of downtown, filtered to Mexican restaurants, might start with 500K documents but only score 200 after filters eliminate the rest. This filter-then-score execution order is the reason multi-faceted search stays fast even on large indexes. ### Facet Aggregations — Counts Alongside Results Beyond returning matching documents, Elasticsearch can compute **facet aggregations** in the same request — counts grouped by category, price range, rating bucket, or any other field. These power the sidebar filters you see on Yelp, Amazon, or any e-commerce site: "Mexican (42), Italian (28), Thai (15)." Facet aggregations run on the **post-filter** result set and are computed in a single pass alongside the main query. This is critical for performance — without aggregation support, you would need separate queries for each facet count, turning one request into dozens. ### Designing the Index for Multi-Faceted Queries The index mapping determines what queries are possible and how fast they run. A common mistake is treating Elasticsearch like a relational database and indexing everything as analyzed text. Instead, use the right field type for each query pattern: A strong interview signal is proposing **multi-field mappings**: index the restaurant name as both `text` (for full-text search) and `keyword` (for exact-match filters and aggregations). This costs extra storage but avoids the need for separate fields, keeping the document structure clean. **See it in practice:** In [Yelp](/learn/system-design/questions/yelp), you combine geo-filtering, keyword search, category facets, and rating sort in a single Elasticsearch query — filter-then-score keeps latency low even across millions of listings. In [LeetCode](/learn/system-design/questions/leetcode), problems are searched by title, filtered by tags and difficulty, and sorted by acceptance rate — facet aggregations power the tag cloud and difficulty distribution sidebar. In [Job Scheduler](/learn/system-design/questions/job-scheduler), search indexes enable filtering scheduled jobs by status, type, and time range. ## Keeping the Search Index in Sync Your primary database (PostgreSQL, MySQL) is the **source of truth**. The search index is a derived, read-optimized copy. The critical design question: how do you keep them in sync? There are three main approaches: **CDC with Debezium is the recommended approach** for most systems. Here is why: - **Non-invasive**: No application code changes. Debezium connects directly to the database replication log (PostgreSQL WAL or MySQL binlog) and captures every INSERT, UPDATE, and DELETE. - **Reliable**: Kafka provides at-least-once delivery. The Elasticsearch consumer is idempotent — it upserts documents by ID, so processing the same event twice is harmless. - **Decoupled**: The primary database has no knowledge of Elasticsearch. You can add or remove search indexes without touching the application. The trade-off is **eventual consistency**: changes appear in search results after a short delay, typically a few seconds. For search, this is almost always acceptable — users do not expect real-time precision from search results the way they expect it from, say, their account balance. **See it in practice:** In [Yelp](/learn/system-design/questions/yelp), CDC syncs restaurant data and reviews to Elasticsearch so new reviews appear in search within seconds. In [Bookstore](/learn/system-design/questions/bookstore), CDC keeps the book catalog searchable as inventory and metadata change. The [Pub/Sub Messaging](/learn/system-design/design-patterns/pub-sub-messaging) pattern covers Kafka's delivery guarantees and consumer group mechanics that make CDC consumers reliable. ## When to Reach for Full-Text Search Not every application needs Elasticsearch. Here is a quick decision framework: The general rule: start with your database's built-in capabilities. Add Elasticsearch only when you need relevance ranking, multi-field search, or query composability that SQL cannot provide. The operational cost of running an Elasticsearch cluster — monitoring, tuning, keeping it in sync — is significant, so make sure the use case justifies it. **See it in practice:** Almost every showoffer system design question involves some form of search. [Yelp](/learn/system-design/questions/yelp) is the canonical full-text + geo search. [LeetCode](/learn/system-design/questions/leetcode) uses search for problem discovery. [Slack](/learn/system-design/questions/slack) indexes messages for keyword search within channels. [Google Maps](/learn/system-design/questions/google-map) uses geospatial indexes for point-of-interest search. In [ChatGPT](/learn/system-design/questions/chatgpt), vector search powers semantic retrieval over conversation history. The [Distributed Caching](/learn/system-design/design-patterns/distributed-caching) pattern pairs naturally with search — cache popular query results in Redis to avoid hitting Elasticsearch on every request.