Sign in to keep reading
“Distributed Cache” is free with an account. Your progress and bookmarks follow you across devices.
Sign in to continueDesign a scalable distributed cache system supporting key-value operations, TTL expiration, eviction policies (LRU/LFU), consistent hashing, replication, and configurable consistency models.
“Distributed Cache” is free with an account. Your progress and bookmarks follow you across devices.
Sign in to continueThink Beyond the Happy Path
Could you answer these today? Tick what you already know.
Designing a distributed cache involves balancing multiple factors—availability, consistency, scalability, latency, and cost—so that you can handle large amounts of data efficiently across multiple nodes.
When NOT to Use a Distributed Cache
Before diving into the design, consider:
A Senior/Staff+ engineer designs for these edge cases from day one.
FR1: Basic Key-Value Operations
Enable set, get, update, and delete operations, optionally supporting bulk operations for efficiency.
FR2: TTL Support
Support TTL (Time-To-Live) for automatic expiration of key-value pairs.
FR3: Configurable Eviction Policies
Implement eviction policies (LRU, LFU) for managing storage limits when memory is exhausted.
FR4: Partitioning & Rebalancing
Provide partitioning options (e.g., consistent hashing) with automatic data and load rebalancing.
FR5: Caching Strategies
Support write-through, write-behind, and read-through caching patterns.
Bonus: Even though partition and replication are usually considered non-functional requirements, due to the nature of this particular problem, we can make consistency and partition configuration part of the functional requirements.
| Requirements Summary | |
|---|---|
| Name | Description |
| FR1 | Basic Key-Value Operations | Set, get, update, delete with optional bulk operations. |
| FR2 | TTL Support | Automatic expiration of key-value pairs with configurable time-to-live. |
| FR3 | Configurable Eviction Policies | LRU, LFU policies for managing memory limits. |
| FR4 | Partitioning & Rebalancing | Consistent hashing with automatic data redistribution. |
| FR5 | Caching Strategies | Write-through, write-behind, and read-through patterns. |
Monitoring is Crucial: For infrastructure-style questions, add monitoring for each functional requirement: hit/miss ratios, latency percentiles, memory usage, eviction rates, and replication lag.
Based on our functional requirements, we need to expose core operations through an API:
// Basic Operations
SET(key, value, ttl?) -> OK | ERROR
GET(key) -> value | NULL
DELETE(key) -> OK | NOT_FOUND
// Bulk Operations
MSET([(key1, value1), (key2, value2), ...]) -> OK | PARTIAL_ERROR
MGET([key1, key2, ...]) -> [value1, value2, ...]
// Atomic Operations
INCR(key, delta) -> new_value
CAS(key, expected_value, new_value) -> OK | CONFLICT
| Cache Entry | |
|---|---|
| Name | Description |
| key (PK) | Unique identifier for the cached value. Typically a string. |
| value | The cached data. Can be serialized objects, strings, or binary. |
| ttl | Time-to-live in seconds. NULL means no expiration. |
expiration_time | Absolute timestamp when the entry expires. |
| version | Monotonic version for CAS operations and conflict detection. |
| NFR1 | Performance | Sub-10ms latency for most requests with efficient concurrency. |
| NFR2 | Scalability | Horizontal scaling to add/remove nodes seamlessly. |
| NFR3 | Fault Tolerance | Replication, automatic failover, multi-AZ/region disaster recovery. |
| NFR4 | Consistency | Configurable strong or eventual consistency models. |
| NFR5 | Atomic Operations | Increment, compare-and-set for data integrity. |
access_time |
| Last access timestamp for LRU eviction. |
access_count | Access frequency counter for LFU eviction. |
created_at | Entry creation timestamp for debugging and metrics. |