Design a real-time multiplayer typing race - 100K concurrent racers sending 1M keystrokes per second, 2 to 8 players per race, a wrong character that freezes one player for one second, and 100K spectators on a single featured race. The staff-level signal is keeping one authoritative in-memory room per race between two small transactions, saying which records commit together and when the ACK goes out, and naming the source of truth a caller checks after a timeout or a crash.
Advanced60 min readPremium
S+
Written by a Staff+ Engineer and an Engineering Manager
12+ years of experience in distributed systems and real-time messaging. Tech lead for large-scale chat applications serving billions of users. Over 12 years of industry experience with 6 years in engineering leadership, building and scaling tech infrastructure teams that deliver end-to-end large-scale distributed systems.
Most candidates can draw the happy path of this system in ten minutes: a matchmaking queue, a game server with WebSockets, a database for results, a leaderboard. The interview is decided by what happens next. The interviewer picks one arrow on your whiteboard and breaks it. These are the six breaks to expect, taken from a real staff-level mock. Each one is a deep dive later on this page.
Swipe to see all columns
The six questions, and the answer that sounds right
#
The interviewer asks
The answer that sounds right
What it misses
DD1
The leaderboard worker adds 7 points for race R42 and crashes before it acknowledges the event. The event is delivered again. How does the player not get 14 points?
Keep a table of processed events. Add the points, then mark the event as processed
The points and the mark are two writes. A crash between them doubles the points. Marking first and crashing loses them
DD2
Player A has a 20 ms round trip and Player B has 120 ms. The server receives A's last key 30 ms before B's. Who wins?
Subtract each player's network delay. B's key travelled 60 ms and A's 10 ms, so B pressed first and B wins
A round trip of 120 ms does not say how long one direction took. And a rule that rewards a slow connection invites players to look slow
DD3
The room receives the winning key and crashes before it sends GAME_END. The players reconnect. Official winner, or void the race?
Nobody was told the result, so void the race and start a rematch
The result may already be committed in the database, with its score event. Voiding now gives the race two answers
DD4
Two matchmaking workers run at once. One proposes A + B, the other A + C. What does each one write?
Compare-and-set A's ticket from PENDING to MATCHED. The worker that loses backs off
A is only half of a match. The partner's ticket, the race and its seats must commit together with A's ticket, or not at all
DD5
A featured race has 100,000 spectators. That is 500,000 deliveries per second for one race. Where does the fan-out happen?
The game server keeps the viewers' connections and broadcasts each update to all of them
The process that judges keystrokes now writes 100,000 sockets five times a second. One slow viewer delays every player's ACK
DD6
The room applies input number 42, then the socket drops before the player sees the ACK. How does the player resume?
After the reconnect, the browser sends again every key that was not acknowledged
Key 42 was applied. Sent again as a new key it is judged a second time, and the old socket can still deliver a late frame
If one of these makes you uneasy, that is the right instinct, and it tells you where to spend your reading time. The same three habits answer all six:
Name what commits together. Say which rows are written in one transaction, and what a reader sees if the process dies one line later.
Say when the ACK or the notification goes out. Before the commit or after it. To the one player who typed, or to everyone on the next tick.
Name the source of truth after a timeout or a crash. When a caller did not get its answer, say which record it reads to find out what really happened.
The rest of the page builds the design step by step, then answers the six questions in order.
You have time for about five questions. Do not spend them on features. Spend them on the rules of the game, because the rules decide the architecture: what exactly happens on a wrong character, what "first" means, and what a player gets back after a disconnect. The answers below are the ones the interviewer gave in the mock.
Swipe to see all columns
Scope Decisions
Question
Agreed answer
Why it matters
How are players matched?
Language is an exact match. Prefer similar skill and a server region with acceptable latency for everyone. Widen the skill band as players wait. Start with at least two ready players, and allow a short window to fill up to eight seats
Matching is also choosing a region, and fairness starts here (DD2, DD4)
What exactly happens on a wrong character?
The cursor stays where it is. The player is frozen for one second. Input during the freeze is ignored, and it does not extend the freeze
The freeze is a rule the server enforces on arrival time (Phase 2)
Out of scope (say this out loud): sign-up and login, the content pipeline that produces passages, anti-cheat analysis of typing patterns, tournaments, payments, and the rendering of the web client. Each is a fair extension question; none should eat the first ten minutes.
Five FRs. FR2 and FR3 share the second phase of the High Level Design. The others map to one phase each.
FR1 — Join a race
A player picks a language and enters the queue with POST /matching/submit. The system groups 2 to 8 ready players by exact language, with compatible skill and acceptable latency to one common region. The player can cancel while queued and can always look up what happened to their request.
FR2 — Race
Everyone in a race gets the same passage and the same start time. The server validates every keystroke, freezes only the player who made a mistake, and shows each racer the progress and errors of the others. The first validated completion wins.
FR3 — Reconnect
A player whose connection drops has 10 seconds to come back to the same seat over a new connection. They get their confirmed position and any freeze that is still running. The race does not pause for them.
FR4 — Watch
A spectator can open a race that is running and follow progress and errors, read-only, including joining halfway through.
FR5 — Result and ranking
When the race ends, the system publishes the official placements and adds each player's points to the seasonal leaderboard of that language.
Tip
The key principle behind all five
Swipe to see all columns
Five facts and the conclusions they do not supportRead each row as: the fact on the left does not, by itself, imply the claim in the middle.
Fact
Does not imply
Why the gap exists
The browser showed the character
The character counts
Only the room's validation moves the cursor. What the browser draws before the ACK is a guess
NFR1 — One authority, one result. Each race has exactly one room that owns it, one order of inputs, and one official result. Gameplay decisions are made by that room. Progress views and leaderboard updates may arrive a moment later.
NFR2 — Latency, with its boundaries. From the moment a player submits a key to the moment the browser receives the room's ACK: p95 ≤ 100 ms, for matches placed in an eligible region. From an accepted key to the opponent's screen showing it: p95 ≤ 300 ms. From entering the queue to an assignment: p95 ≤ 10 s in well-populated queues. Spectator lag: p95 ≤ 1 s, also during a featured-race burst. From the result commit to the visible leaderboard: p95 ≤ 5 s in healthy operation.
NFR3 — Scale. 100K concurrent racers typing 10 characters per second is 1M keystrokes per second. 20K viewers are spread over ordinary races. One featured race can draw 100K viewers at once.
NFR4 — Isolation. A spectator spike never slows a player's ACK. The two kinds of traffic have separate capacity.
NFR5 — Recovery. A player gets the same seat back within 10 seconds over a new connection. A room that crashes leaves either the committed result or an explicit void, never both. A race is counted on the leaderboard exactly once.
100K ÷ 5 racers on average = 20K. The upper bound is 50K, if every race had only two players
Keystrokes in
100K racers × 10 keys per second = 1M events per second
Room snapshots
20K rooms × 5 per second = 100K snapshots per second
Progress deliveries, normal peak
(100K racers + 20K viewers) × 5 per second = 600K per second, plus the ACK traffic
The number that identifies the real system is not 1M keystrokes per second. It is the gap between three numbers:
Code
keystrokes in 1,000,000 / s judged in the room's memory, acknowledged at once
progress out 600,000 / s one snapshot per room, every 200 ms
official results ~220 / s the only durable write on the game path
A million events per second arrive, and about two hundred transactions per second leave. Everything in between lives in memory, in the one room that owns the race. A design that puts a database on the keystroke path has to write 1M rows per second for a game that lasts two minutes. Every number above is an illustrative assumption to be replaced by measured traffic.
Interviewer signal: candidates who send the ACK together with the progress broadcast have missed a number. A broadcast 5 times a second means an update can wait almost 200 ms for the next tick, which already breaks the 100 ms target. Candidates who say "the ACK goes back to the typer at once, the broadcast waits for the tick" have found the system.
The one-second freeze is a rule of the game. If the browser enforced it, a player could remove it with a modified client. So the server decides everything that affects the result: which character is expected, whether a key was right, when a freeze ends, who finished first. The browser may draw a guess to feel fast, and it corrects itself when the ACK arrives. That is why each race has one room, on one server, with one order of inputs.
Keystrokes need an answer in milliseconds, so the room keeps each seat's live state in memory. A result has to survive a crash, so it is written to PostgreSQL. The design is a long stretch of in-memory work between two small transactions: one that creates the race and its seats, and one that ends it. The price is stated openly. If the room's server dies in the middle of a race, the live state is gone and the race is void.
3. Racers and spectators are two different systems#
Eight racers need an ACK within 100 ms. A hundred thousand viewers are content with a picture that is up to a second old. Serve both from the same loop and the larger group slows the smaller one. Viewers get their own gateways, their own capacity and a path the room only publishes to.
Tip
The core recognition
This is one authoritative in-memory room per race, placed between two small transactions. The first transaction claims the tickets and creates the race and its seats. The second writes the final status, the placements and the score event. Everything that happens between them is fast, lives in one process, and is allowed to be lost. Everything on either side of them is durable, and can be looked up by anyone who missed a message.
Design Decisions
Design choice
Reason
A ticket is a durable row, created with an idempotency key
A player who lost the response can ask again and get the same ticket, and can always look up their assignment
Claiming the tickets and creating the race and its seats is one transaction
Six entities carry the design. Spend your explanation on Race, RaceSeat and LiveSeatState: which of them is durable, which one lives only in the room, and which fields move on every keystroke, is most of the interview.
MatchTicket
One player's request to race. It is the record a player looks up when a message was lost.
Swipe to see all columns
MatchTicket
Field
Type
Comment
ticket_id
STRING
Primary key, for example T-A
user_id, language
STRING
Who is waiting, and for which language queue
Race
One race from the match to the official result.
Swipe to see all columns
Race
Field
Type
Comment
race_id
STRING
Primary key, for example R42
language, region
STRING
The queue it came from and the region its room runs in
RaceSeat
One player's place in one race. Created with the race, completed with the result.
Swipe to see all columns
RaceSeat
Field
Type
Comment
race_id, user_id
STRING
Primary key together
seat_status
ENUM
SEATED
LiveSeatState
What the room knows about one seat right now. It lives only in the room's memory.
Swipe to see all columns
LiveSeatState
Field
Type
Comment
race_id, user_id
STRING
Which seat
cursor
INT
How many correct characters the player has typed. It never moves on a wrong or ignored key
SeasonScore and the score ledger
A player's total for one season and language, and the proof of which races are in it.
Swipe to see all columns
SeasonScore and ScoreLedger
Field
Type
Comment
season_id, language, user_id
STRING
Primary key of SeasonScore
total_points
INT
Three supporting records make the entities above safe to operate:
Swipe to see all columns
Supporting records
Record
What it holds
Why it exists
User
user_id, display name, a skill estimate per language
Matchmaking reads the skill. It is a separate number from seasonal points (DD1)
Outbox rows
A message to send, stored in the same transaction as the change it announces: race created, score event, new season total
A relay reads these rows and delivers them until they are acknowledged. A change is never committed without its message (DD1, DD4)
Swipe to see all columns
Four small state machines
Record
States
Rule
MatchTicket
PENDING, then MATCHED, CANCELLED or EXPIRED
A ticket leaves PENDING once. A cancel that arrives after the match returns the assignment instead
Race
ACTIVE, then FINALIZED or VOID
Both endings are conditional updates on the same row. The first to commit wins, the other changes nothing (DD3)
RaceSeat
SEATED, then FINISHED or DNF
Written by the result transaction, together with the race status
There is no durable record of a keystroke. The room judges a key, updates four numbers in memory and answers. What becomes durable is the outcome of the race.
Strong, transactional. The idempotency key, the ticket claim, the race ending and the score ledger all rely on it
Room memory
LiveSeatState for every seat of the race, the passage, the race clock
One process, one writer. Lost if the server dies. The baseline then voids the race (DD3)
Pub/Sub, one topic per race
The latest race view snapshot on its way to the spectator gateways
Who wins?
The first completion that the server has validated. The others get up to 10 seconds to place, capped at two minutes from the start. Whoever has not finished by then is marked DNF, did not finish
The server orders arrivals, not key presses (DD2). The result is one transaction (DD3)
Can a player reconnect?
Yes, within 10 seconds, to the same seat, with their confirmed progress and any remaining freeze. One active connection per player. The race clock keeps running
The server must be able to tell a client where it is (DD6)
How big?
100K concurrent racers and 20K viewers across regions at peak. 100K viewers on one featured race is a separate burst scenario
1M keystrokes per second, and a fan-out that must not touch the players (DD5)
What does the leaderboard track?
Seasonal points per language, awarded by placement. A player who does not finish gets zero. Matchmaking uses a separate skill estimate
Points grow with every race played. Skill must not, or playing more would move a player into a harder band (DD1)
A pressed the key before B
A's key arrived before B's
The two keys travel over different networks. The server can only order what it receives
The room knows who won
The result is official
The room's memory is lost in a crash. A result is official when its transaction has committed
GAME_END never arrived
The race has no result
The commit happens before the message. The message can be lost after the commit
The score event was delivered
It was delivered once
A worker can crash after adding the points and before acknowledging. The event comes again
Most of the design exists to keep those five distinctions honest.
One featured race
100K viewers × 5 per second = 500K deliveries per second, for a single room
Race completions
20K rooms ÷ 90 s average race ≈ 220 per second. The 90 s is a capacity assumption, the cap stays two minutes
A player never has two races, and a race never has a missing seat
One room per race, owned by one server under a lease with an epoch number
One order of inputs, no locks on the keystroke path, and a stale owner can be shut out later
Live seat state lives in the room's memory
A million keystrokes per second never touch a database
Every input carries a sequence number and a connection generation
A duplicate is recognised, and a frame from an old socket cannot change the seat
The ACK goes to the typer at once, progress goes to everyone on a tick
The 100 ms target does not wait for a 200 ms tick
The final status, the placements and the score outbox are one transaction, and GAME_END is sent after it
There is exactly one official result, and nobody is told a result that does not exist
Finalizing and voiding are conditional updates on the same race row
Only one of the two can win, whatever crashed
Spectators get their own gateways behind a pub/sub topic per race
The room publishes once per tick, however many people watch
Ledger row, season total and projection are one transaction; Redis receives absolute totals
A score event that arrives twice counts once, and the ranking can be rebuilt
skill
INT
The player's skill estimate for this language, copied when they enter the queue
region_latencies
JSON
The round-trip time from this browser to each candidate region, measured by a probe before queueing (DD2)
idempotency_key
STRING
Unique. A retried submit returns the same ticket (DD4)
queued_at, expires_at
TIMESTAMP
The queue position, and when the ticket stops being valid
status
ENUM
PENDING / MATCHED / CANCELLED / EXPIRED
race_id
STRING
Empty until the ticket is matched, then written once
passage
TEXT
The exact text, copied onto the row. It never changes after the race is created
season_id
STRING
Which season the points count for, fixed at creation (DD1)
start_at
TIMESTAMP
The common start time
status
ENUM
ACTIVE / FINALIZED / VOID
owner_epoch, lease_until
INT, TIMESTAMP
Which server owns the room, as a number that grows with every takeover, and until when (DD3)
winner_id
STRING
Empty until the result commits
/
FINISHED
/
DNF
finished_at
TIMESTAMP
The room's validated finish time
final_place
INT
1 for the winner. Written by the result transaction
points_awarded
INT
The points of this race under the season's scoring rule
error_count
INT
How many wrong characters so far
frozen_until
TIME
The room time at which the current freeze ends
last_processed_seq
INT
The number of the last input the room handled. Every input counts, also a wrong or ignored one
connection_generation
INT
Grows with every new socket for this seat. Only the newest may write (DD6)
The sum of all points this season
version
INT
Grows with every change. Lets a reader tell a newer total from an older one (DD1)
ScoreLedger: race_id, user_id
STRING
Unique together. One row per player per race. This row is what stops a second count
ScoreLedger: points
INT
What that race added
Seat token
A short-lived signed credential: race, user and expiry
The gateway can check that a socket belongs to a seat without asking the database on every connection
Seat connection
Generation 1, 2, 3 and so on
A new socket takes the next generation. Frames from an older generation are rejected (DD6)
Best effort and replaceable. A missed snapshot is covered by the next one (DD5)
Redis sorted sets
One ranking per season and language
A projection of SeasonScore. It can be rebuilt from PostgreSQL at any time (DD1)
Browser
Keys the player typed that the room has not acknowledged yet
A guess. Dropped on a reconnect, when the room says where the player really is (DD6)