Sign in to keep reading
“Scaling Writes” is free with an account. Your progress and bookmarks follow you across devices.
Sign in to continueNEW: ML Mock & Coaching with a 20+ year experience coach. Save $20 on sessions for Premium.ML Mock & Coaching now available
NEW: ML Mock & Coaching with a 20+ year experience coach. Save $20 on sessions for Premium.ML Mock & Coaching now available
Strategies for handling write-heavy workloads including sharding, consistent hashing, write queues, and LSM-tree storage engines.
“Scaling Writes” is free with an account. Your progress and bookmarks follow you across devices.
Sign in to continueA well-tuned PostgreSQL instance can handle roughly 10,000 writes per second. That sounds like a lot — until you consider that Uber processes millions of driver location updates per minute, or that the Ad Click Aggregator ingests millions of click events per second during a Super Bowl ad campaign. When reads are the bottleneck, you can throw caches and replicas at the problem (see Scaling Reads). But writes are harder — every write must reach the source of truth, and there's no "write cache" that magically solves it.
The good news: there's a clear toolkit for scaling writes, and each tool addresses a different part of the problem. A combination of four strategies will allow you to scale writes beyond what a single, unoptimized database or server can handle: vertical scaling and database choices, sharding and partitioning, handling bursts with queues and load shedding, and batching with hierarchical aggregation.
Before splitting your data across machines, make sure you've exhausted the hardware at your disposal. We want to show our interviewer that we've done our due diligence and we're not prematurely adding complexity when hardware or local software tweaks will do it.
Writes are bottlenecked by disk I/O, CPU, or network bandwidth. Before re-architecting, confirm you're actually hitting those walls. Often this means doing some quick back-of-the-envelope math to check (a) what your write throughput actually is, and (b) whether that fits within your hardware capabilities.
Many candidates assume instances with 4-8 cores and a single spinning-platter hard disk. But cloud providers routinely offer systems with 200+ CPU cores and 10-gigabit network interfaces. Hardware can go further than you think — it's worth brushing up on your numbers to make sure you're not missing potential.
You can make the case (and most interviewers will appreciate it) that some of your challenge is solved with modern hardware. However, many interviewers have an assessment built around non-vertical scaling, and they'll frequently move the goalposts until you're forced to contend with scale in other ways. So make the case, but try not to get into a back-and-forth if the interviewer isn't receptive.
Most general-purpose databases are designed for mixed workloads — reads and writes. When you know your system is write-heavy, you can make targeted choices that dramatically improve performance.
Databases like Cassandra, RocksDB, and ScyllaDB use LSM-trees (Log-Structured Merge Trees) instead of B-Trees. The difference in write performance is dramatic:
The trick: writes never update data in place on disk. They're always either in memory or appended sequentially. Sequential I/O is orders of magnitude faster than random I/O on both SSDs and spinning disks.
The trade-off: reads are slower because data might be spread across multiple SSTables. LSM-trees trade read performance for write performance — which is exactly what you want in a write-heavy system.
Other databases make similar trade-offs:
| Write-Optimized Database Options | ||
|---|---|---|
| Database Type | Why It's Fast for Writes | Trade-off |
| LSM-Tree (Cassandra, RocksDB) | Append-only commit log, sequential I/O | Reads check multiple SSTables — slower lookups |
| Time-Series (InfluxDB, TimescaleDB) | Built for high-volume sequential writes with timestamps | Limited query flexibility outside time-range patterns |
| Column Store (ClickHouse) | Batch writes efficiently for analytics workloads | Not suited for transactional point updates |
Beyond choosing the right database, you can optimize any database for writes: disable expensive features like foreign key constraints or complex triggers during high-write periods, tune write-ahead logging to batch multiple transactions before flushing to disk, and reduce index overhead (fewer indexes mean faster writes, though you'll pay for it on reads).
In interviews, mentioning specific database choices shows you understand the trade-offs. Don't just say "use a faster database" — explain why Cassandra's append-only writes are faster than MySQL's B-tree updates, or why you might choose a time-series database for metrics collection.
See it in practice: In the Ad Click Aggregator, Cassandra (LSM-tree) stores raw click events because the write rate is too high for a B-Tree database. In the Leaderboard, Redis sorted sets act as an in-memory write-optimized store for rapid score updates. The Time-Series & Metrics Ingestion pattern covers how time-series databases use LSM-tree variants optimized for append-heavy workloads.
| Appends new data rather than updating in place |
| Read amplification during compaction |