Almost every system design question involves caching somewhere. In the [Distributed Cache](/learn/system-design/questions/cache) question, you design a full Redis-like system from scratch. In [Leaderboard](/learn/system-design/questions/leaderboard), Redis Sorted Sets power real-time rankings. In [Uber](/learn/system-design/questions/uber), Redis GEO commands find nearby drivers. The pattern keeps showing up because a single-server in-process cache has two fatal problems: it disappears on restart, and it is not shared across multiple application servers. A **distributed cache** solves both. It runs as a separate, shared tier that every app server can access over the network. Think of it as a fast, shared scratchpad sitting between your application and your database. When it works well, the database barely notices traffic at all. ## How the Architecture Works The core flow is simple. Your app checks the cache first. If the key exists (**cache hit**), it returns in under a millisecond — the database is never touched. If the key is missing (**cache miss**), the app falls through to the database, gets the data, and writes it into the cache so the next request is fast. Here is the key insight: in a well-tuned system, 95%+ of reads are cache hits. That means your database handles only a fraction of the actual traffic. This is exactly how the [Scaling Reads](/learn/system-design/design-patterns/scaling-reads) layered caching funnel works — CDN handles static assets, Redis handles dynamic data, and the database sees only what falls through both layers. There are three main strategies for keeping the cache and database in sync: **Cache-aside is the default choice.** It is simple, resilient (cache failure means slower, not broken), and naturally adapts to access patterns — only frequently-read data stays cached. **See it in practice:** The [Distributed Cache](/learn/system-design/questions/cache) question walks you through designing a full caching system with these invalidation strategies. The [Scaling Reads](/learn/system-design/design-patterns/scaling-reads) pattern shows where caching fits in the broader read-scaling progression. In [Yelp](/learn/system-design/questions/yelp), restaurant data is cached with cache-aside to avoid repeated database lookups on every page view. ## Redis Data Structures — More Than Key-Value Most people think of Redis as a simple key-value store, but its rich **data structures** are what make it so powerful for system design. Each structure is purpose-built for specific access patterns: The key takeaway: when you hear "cache" in a system design interview, don't just think GET and SET. Ask yourself which data structure fits the access pattern. A [Leaderboard](/learn/system-design/questions/leaderboard) naturally maps to Sorted Sets. An [Uber](/learn/system-design/questions/uber) driver location system naturally maps to GEO commands. A [Slack](/learn/system-design/questions/slack) unread-count tracker maps to Hashes. **See it in practice:** In [Leaderboard](/learn/system-design/questions/leaderboard), ZADD and ZRANGEBYSCORE on a Sorted Set power real-time ranking at scale. In [Uber](/learn/system-design/questions/uber), GEOADD and GEORADIUS find nearby drivers within milliseconds. In [Distributed Counters](/learn/system-design/questions/distributed-counters), atomic INCR on Strings handles high-throughput counting without race conditions. ## Distributing Keys Across Nodes — Consistent Hashing A single Redis instance can handle roughly 100,000 operations per second. That sounds like a lot until you are building something like a [Bookstore](/learn/system-design/questions/bookstore) product catalog cache or a [Facebook Post Privacy](/learn/system-design/questions/facebook-post-privacy) feed cache with millions of users. You need multiple cache nodes. The naive approach — hash(key) % N where N is the number of nodes — has a fatal flaw. When you add or remove a node, N changes, and **every single key rehashes to a different node**. Your entire cache is effectively invalidated at once, causing a massive stampede to the database. **Consistent hashing** fixes this. Keys and nodes are placed on a virtual ring. Each key maps to the next node clockwise on the ring. When a node is added or removed, only the keys between the affected node and its neighbor need to move — roughly `1/N` of total keys instead of all of them. In practice, most production systems use **virtual nodes** (vnodes) — each physical node gets multiple positions on the ring. This smooths out the distribution so one node does not accidentally end up with a disproportionate share of keys. **See it in practice:** The [Distributed Cache](/learn/system-design/questions/cache) question has you design consistent hashing with virtual nodes from scratch. The [Rate Limiting](/learn/system-design/design-patterns/rate-limiting) pattern relies on consistent hashing to route rate-limit counters to the correct cache shard. In [Ads Click Aggregator](/learn/system-design/questions/adds-click-aggregator), consistent hashing distributes click counters across cache nodes to prevent any single node from becoming a bottleneck. ## Eviction — What Happens When the Cache Is Full Memory is finite. When your cache fills up, it needs to decide which entries to throw away to make room for new ones. This is the **eviction policy**, and choosing the right one matters. **LRU is the default choice for Redis** (allkeys-lru policy). It naturally adapts to changing access patterns — entries that stop being popular drift to the bottom and get evicted. LFU is better when you have a small set of keys that are always hot (like a top-10 leaderboard), because it protects frequently-accessed keys even if they were not accessed in the last few seconds. One important detail: Redis does not implement true LRU. It uses **approximate LRU** — it samples a configurable number of random keys (default 5) and evicts the least recently used among those. This is much more memory-efficient than maintaining a full LRU linked list, and in practice the approximation is very close to true LRU. **See it in practice:** In [Distributed Cache](/learn/system-design/questions/cache), you implement eviction policies as a core part of the cache design. In [Top-K](/learn/system-design/questions/topk), LFU-style eviction ensures the most frequently queried items stay cached. In [ChatGPT](/learn/system-design/questions/chatgpt), conversation history caching uses TTL-based eviction since old conversations naturally expire. ## High Availability — Surviving Node Failures A cache that goes down under load can be worse than no cache at all. Without the cache absorbing traffic, every request falls through to the database simultaneously — a phenomenon called **cache avalanche**. You need your cache to survive node failures. Redis offers two high-availability architectures: **Redis Sentinel** provides automatic failover for a single-shard setup. Sentinel processes monitor the primary node. If the primary goes down, Sentinels elect a new primary from the replicas and update all clients with the new address. The failover happens in seconds, and clients using a Sentinel-aware library reconnect automatically. **Redis Cluster** provides both high availability and horizontal scaling. Data is partitioned across multiple shards using 16,384 **hash slots**. Each shard has its own primary and replicas. If a shard's primary fails, its replica is promoted — just like Sentinel, but per-shard. Clients use a cluster-aware library that routes each command to the correct shard. **See it in practice:** In [Distributed Cache](/learn/system-design/questions/cache), you design replication and failover from scratch. In [Payment](/learn/system-design/questions/payment), cache availability is critical — a cache outage during checkout could either slow transactions or lose session data. The [Pub-Sub Messaging](/learn/system-design/design-patterns/pub-sub-messaging) pattern shows how Redis Pub/Sub can propagate cache invalidation events across nodes. ## Common Problems and How to Solve Them ### Hot Key Problem Sometimes one key gets wildly more traffic than everything else — a trending post in [Facebook Post Privacy](/learn/system-design/questions/facebook-post-privacy), a live [Leaderboard](/learn/system-design/questions/leaderboard) during a competition, or a viral product page. Even with consistent hashing across many nodes, all requests for that one key hit the same node. Redis is single-threaded, so that node becomes the bottleneck. Three solutions, from simplest to most involved: - **Local in-process cache (L1)**: Each app server caches the hottest keys in its own memory. This eliminates the network round-trip to Redis entirely. Use a very short TTL (a few seconds) so data stays reasonably fresh. This is the most effective solution for read-heavy hot keys. - **Key replication with suffixes**: Store the same value under trending:post:0, trending:post:1, ..., trending:post:N. Each read randomly picks a suffix. This spreads load across N Redis nodes instead of concentrating on one. - **Request coalescing (singleflight)**: If 100 requests arrive for the same key at the same moment, only one actually fetches it from Redis. The other 99 wait and share the result. This is built into some frameworks (Go's `singleflight` package, for example). **See it in practice:** In [Facebook Post Privacy](/learn/system-design/questions/facebook-post-privacy), a viral post can generate millions of reads on a single cache key — L1 caching and key replication prevent Redis from becoming the bottleneck. In [Distributed Counters](/learn/system-design/questions/distributed-counters), the entire problem revolves around hot-key contention at scale. In [Top-K](/learn/system-design/questions/topk), the most popular items are inherently hot keys that need special handling. ### Cache Stampede (Thundering Herd) Imagine a popular cached item expires. Suddenly, thousands of requests see a cache miss at the same time and all hit the database simultaneously. The database, which was handling a trickle of cache misses, is now flooded. This is a **cache stampede** — and it can cascade into a full outage if the database falls over under the load. Three solutions: - **Probabilistic early refresh**: Instead of all readers waiting until the TTL expires, each reader has a small random chance of refreshing the key *before* it expires. This spreads recomputation over time so there is no sudden cliff. - **Lock on recomputation**: When a cache miss happens, the first request acquires a distributed lock (using Redis SET NX with a TTL) and recomputes the value. All other requests either wait for the lock to release or serve slightly stale data. This guarantees only one request hits the database. - **Background refresh**: A background job proactively refreshes popular entries before they expire. No user request ever triggers a recomputation — the cache is always warm. This is the most robust solution but requires knowing which keys are popular. **See it in practice:** In [Distributed Cache](/learn/system-design/questions/cache), you design stampede prevention directly into the cache layer using locking and background refresh. In [Ads Click Aggregator](/learn/system-design/questions/adds-click-aggregator), cached click counts for popular ads expire simultaneously during peak traffic — probabilistic early refresh prevents the thundering herd. The [Dealing with Contention](/learn/system-design/design-patterns/dealing-with-contention) pattern covers the broader problem of concurrent access to shared resources. ### Cache Warming When you deploy a new cache cluster or restart after a failure, the cache is empty — every request is a miss. This **cold start** problem can overwhelm the database with a sudden flood of traffic. The fix: **pre-warm** the cache before routing traffic to it. Load the most frequently accessed keys from the database (or from a snapshot of the old cache) before the new node starts serving requests. In Redis, you can use RDB or AOF persistence to restore state on restart, so the cache comes back warm instead of empty. **See it in practice:** In [Job Scheduler](/learn/system-design/questions/job-scheduler), the job queue cache must be warm before the scheduler starts dispatching work — a cold cache would cause duplicate job execution. In [Google Calendar](/learn/system-design/questions/google-calendar), calendar data for active users is pre-warmed into the cache during deployments to avoid a thundering herd on the database.