Understanding search infrastructure, Elasticsearch, and when to use dedicated search systems in system design.
Finished this lesson?
Sign in to save your progress across devices.
Tip: use ← and → to move between lessons.
Was this lesson helpful?
Rehearse this with a Staff+ engineer
45-minute mock with a Staff+ engineer who interviews at your target company. Premium members save $20.
Search systems enable fast, flexible queries over large datasets. When your database's LIKE queries aren't enough, dedicated search infrastructure becomes essential.
Database limitations:
Search systems provide:
If users need to search through text content—products, articles, users, logs—you likely need a search system.
| Popular Search Solutions | |
|---|---|
| Name | Description |
| Elasticsearch | Industry standard. Powerful, flexible, great ecosystem. Complex to operate. Best for: general search, logging, analytics. |
| OpenSearch | AWS fork of Elasticsearch. Similar features, different licensing. Best for: AWS-native applications. |
| Algolia | Managed search-as-a-service. Easy to use, fast. Higher cost at scale. Best for: e-commerce, simple use cases. |
| Meilisearch | Open source, easy to set up. Good for smaller scale. Best for: startups, simpler needs. |
| Typesense | Open source, typo-tolerant. Fast, easy. Best for: applications needing typo tolerance. |
The key data structure for fast search.
Traditional index:
Document 1: "The quick brown fox"
Document 2: "The lazy dog"
Inverted index:
"the" → [Doc 1, Doc 2]
"quick" → [Doc 1]
"brown" → [Doc 1]
"fox" → [Doc 1]
"lazy" → [Doc 2]
"dog" → [Doc 2]
Searching "quick fox" → intersect posting lists → Document 1
{
"index": "products",
"document": {
"id": "123",
"title": "Wireless Bluetooth Headphones",
"description": "High-quality wireless headphones with noise cancellation",
"category": "electronics",
"price": 149.99,
"rating": 4.5,
"tags": ["wireless", "bluetooth", "noise-cancelling"]
}
}
Transform text for indexing and searching.
Input: "Running Shoes for Women"
Analyzer steps:
1. Lowercase: "running shoes for women"
2. Tokenize: ["running", "shoes", "for", "women"]
3. Stop words: ["running", "shoes", "women"] (removed "for")
4. Stemming: ["run", "shoe", "woman"]
Final tokens indexed: ["run", "shoe", "woman"]
| Common Analyzer Components | |
|---|---|
| Name | Description |
| Tokenizer | Splits text into tokens. Standard (word boundaries), whitespace, n-gram. |
| Lowercase filter | Converts to lowercase for case-insensitive search. |
| Stop words filter | Removes common words (the, a, is) that don't add meaning. |
| Stemmer | Reduces words to root form (running → run). |
| Synonym filter | Expands queries with synonyms (couch → sofa). |
Application → Search Service → Elasticsearch
↘ Primary Database (source of truth)
| Sync Strategies | |
|---|---|
| Name | Description |
| Dual write | Write to both database and search. Simple but can get out of sync on failures. |
| Change Data Capture (CDC) | Capture database changes, stream to search. Reliable, eventually consistent. |
| Periodic reindex | Rebuild index on schedule. Simple, but stale between rebuilds. |
| Event-driven | Publish events on changes, consumer updates search. Decoupled, scalable. |
Recommended: CDC (using Debezium) or event-driven for production systems.
Design a Product Search System
You're building product search for an e-commerce site with 10 million products. Users search by name, filter by category and price, and sort by relevance or price. How would you design it?
Architecture:
User search → API Server → Elasticsearch
↘ PostgreSQL (product details)
Product updates → Kafka → Search Indexer → Elasticsearch
Index design:
{
"mappings": {
"properties": {
"name": { "type": "text", "analyzer": "standard" },
"description": { "type": "text" },
"category": { "type": "keyword" },
"price": { "type": "float" },
"rating": { "type": "float" },
"in_stock": { "type": "boolean" },
"brand": { "type": "keyword" },
"tags": { "type": "keyword" }
}
}
}
Query strategy:
name and descriptioncategory, range, {
"query": {
"match": {
"title": "wireless headphones"
}
}
}
{
"query": {
"bool": {
"must": { "match": { "title": "headphones" } },
"filter": [
{ "term": { "category": "electronics" } },
{ "range": { "price": { "lte": 200 } } }
]
}
}
}
{
"query": {
"match": {
"title": {
"query": "headhpones",
"fuzziness": "AUTO"
}
}
}
}
{
"aggs": {
"categories": {
"terms": { "field": "category" }
},
"price_ranges": {
"range": {
"field": "price",
"ranges": [
{ "to": 50 },
{ "from": 50, "to": 100 },
{ "from": 100 }
]
}
}
}
}
Returns: counts per category, counts per price range.
{
"suggest": {
"product-suggest": {
"prefix": "wire",
"completion": {
"field": "suggest"
}
}
}
}
Default relevance scoring based on:
Boost results based on business logic:
{
"query": {
"function_score": {
"query": { "match": { "title": "headphones" } },
"functions": [
{
"field_value_factor": {
"field": "rating",
"factor": 1.2
}
},
{
"filter": { "term": { "is_promoted": true } },
"weight": 2
}
]
}
}
}
| Search System Knowledge by Level | |
|---|---|
| Name | Description |
| Mid-Level (L4) | Know when to use search vs database. Understand basic full-text search concepts. Can explain inverted indexes at high level. |
| Senior (L5) | Design index schema for use case. Implement sync strategy (CDC, events). Configure relevance scoring. Handle scaling. |
| Staff+ (L6+) | Design search infrastructure at scale. Optimize query performance. Handle multi-tenant search. Consider search quality metrics. |
| Engineering Manager | Evaluate build vs buy (Elasticsearch vs Algolia). Plan team expertise needs. Understand search relevance impact on business. |
Search systems are eventually consistent and not designed for transactional workloads.
Do: Primary database for writes, search index for reads Don't: Use Elasticsearch as your only data store
Don't index fields you won't search or filter on.
Do: Index searchable fields, store the rest Don't: Index entire documents "just in case"
Schema matters in search. Define mappings explicitly.
{
"mappings": {
"properties": {
"category": { "type": "keyword" }, // Explicit: exact match
"title": { "type": "text" } // Explicit: full-text search
}
}
}
Don't: Rely on dynamic mapping in production
Search is never "done." Measure and iterate.
Search relevance is both art and science. Start simple, measure results, and iterate. Perfect relevance on day one isn't realistic.
"I'd use Elasticsearch as a dedicated search layer. Products would be indexed with searchable fields—name, description, category, tags. The primary PostgreSQL database remains the source of truth. Updates flow through Kafka to a search indexer that updates Elasticsearch. Search queries hit Elasticsearch for fast, relevant results. For the 10 million products scale, a single Elasticsearch cluster with replicas would handle it."
"I'd use change data capture with Debezium. It captures all changes from the PostgreSQL WAL and publishes to Kafka. A search indexer service consumes these events and updates Elasticsearch. This is eventually consistent—a few seconds delay—but reliable. If we need faster sync for critical updates, we could also do a synchronous index update for those specific cases."
"Elasticsearch has built-in fuzzy matching. I'd configure the search query with fuzziness set to AUTO, which allows for 1-2 character edits based on word length. For autocomplete, I'd use edge n-grams to match partial inputs. Additionally, I could maintain a custom dictionary of common misspellings in our domain."
You've now covered the core building blocks for system design. Practice combining them to solve real interview problems in the Questions section.
| Solr | Apache project, mature. More complex than Elasticsearch. Best for: legacy systems, specific enterprise needs. |
pricein_stockrating for relevanceSync strategy:
Scaling: