When you open [Slack](/learn/system-design/questions/slack) and see a message appear the instant someone sends it, or watch a chess piece move in real time during an [Online Chess](/learn/system-design/questions/online-chess) match, or see a [Leaderboard](/learn/system-design/questions/leaderboard) update live during a competition — those aren't magic. Behind the scenes, the server is **pushing** data to your browser the moment something changes, instead of waiting for you to ask for it. The fundamental question is simple: how does the server tell the client that something new happened? The answer shapes the architecture of every real-time feature you'll ever build. Get it wrong and you'll waste bandwidth polling for changes that haven't happened. Get it right and updates arrive instantly with minimal overhead. ## Why Polling Falls Short The naive approach is **polling** — the client asks the server "anything new?" every few seconds. It works, but it's wasteful. Most responses come back empty ("no, nothing changed"), and there's always a delay between when data changes and when the client finds out (up to the full poll interval). **Long polling** is a step up: the client sends a request, and the server holds it open until there's new data (or a timeout). This eliminates empty responses but still has overhead — each response requires a new HTTP request, and the server must hold connections open. For low-frequency updates where simplicity matters more than latency — say, checking for new [CI/CD](/learn/system-design/questions/cicd) pipeline results every 30 seconds — polling or long polling is perfectly fine. But for anything interactive, you need a persistent connection. ## Protocol Choices: WebSockets vs SSE There are two main protocols for server-initiated push, and picking the right one depends on whether communication needs to flow in both directions. ### WebSockets — Bidirectional Communication **WebSockets** upgrade an HTTP connection into a persistent, bidirectional TCP channel. Both client and server can send messages at any time without the overhead of HTTP headers on each message. The connection starts as a normal HTTP request with an Upgrade: websocket header. The server responds with 101 Switching Protocols, and from that point on, the connection is a raw TCP pipe. Messages are tiny — just a few bytes of framing overhead compared to hundreds of bytes for HTTP headers. Use WebSockets when the client needs to send data back frequently: chat messages in [Slack](/learn/system-design/questions/slack), game moves in [Online Chess](/learn/system-design/questions/online-chess), collaborative cursor positions in a [Remote DevBox](/learn/system-design/questions/remote-devbox), or real-time prompts in [ChatGPT](/learn/system-design/questions/chatgpt). **See it in practice:** The [Slack](/learn/system-design/questions/slack) question is the canonical WebSocket design — persistent connections carry messages, typing indicators, and presence updates bidirectionally. The [Online Chess](/learn/system-design/questions/online-chess) question requires WebSockets for sub-second game state synchronization between two players. The [Remote DevBox](/learn/system-design/questions/remote-devbox) question uses WebSockets for terminal I/O streaming. ### Server-Sent Events (SSE) — Unidirectional Push **Server-Sent Events** use a plain HTTP connection that the server keeps open and writes to over time. The client simply calls new EventSource('/events') and receives a stream of messages. Unlike WebSockets, communication is one-way: server to client only. SSE has a killer feature that WebSockets lack: **built-in reconnection with replay**. Each event can include an `id` field. If the connection drops, the browser automatically reconnects and sends a Last-Event-ID header. The server can then replay all events the client missed — no custom reconnection logic needed. Use SSE when the server does most of the talking: live score feeds for a [Leaderboard](/learn/system-design/questions/leaderboard), notification streams, stock price tickers, or [CI/CD](/learn/system-design/questions/cicd) build log streaming. The client can still send data via regular HTTP requests alongside the SSE connection. **See it in practice:** The [Leaderboard](/learn/system-design/questions/leaderboard) question uses SSE to stream ranking updates — the server pushes new positions and the client renders them, with no need for the client to send data back on the same channel. The [ChatGPT](/learn/system-design/questions/chatgpt) question uses SSE for token-by-token response streaming from the language model. ### Choosing Between Them A practical rule: **start with SSE** unless you need the client to send frequent messages over the same connection. SSE is simpler to implement, works through all HTTP infrastructure, and handles reconnection for free. Switch to WebSockets when you genuinely need bidirectional communication. ## Scaling to Millions of Connections A single server can hold tens of thousands of open WebSocket or SSE connections (modern Linux can handle 100K+ with tuning). But when you have millions of connected users, you need multiple **connection servers** — and that introduces a routing problem: when a message is published on Server A, how does it reach a user connected to Server B? ### Connection Management Architecture The architecture has three key components: - **Connection servers** hold the actual WebSocket/SSE connections. Each server tracks which users are connected to it locally. When a server needs to push a message to a user, it looks up the local connection and writes to it. - **Connection registry** (typically Redis or a dedicated service) maintains a global mapping of user-to-server. When you need to send a message to Alice, you look up which connection server she's on and route the message there. - **Presence service** tracks who is online by monitoring heartbeats. If a client stops sending heartbeats (network failure, app closed), the presence service marks them offline and cleans up the registry entry. **See it in practice:** The [Slack](/learn/system-design/questions/slack) question requires this exact architecture — connection servers, a registry for routing direct messages, and a presence service that powers the green/yellow/offline indicators next to each user's name. The [Online Chess](/learn/system-design/questions/online-chess) question uses a simpler variant where only two players per game need connection tracking. ### Fan-Out with Pub/Sub The connection registry solves point-to-point delivery (send a message to one specific user). But what about group channels in [Slack](/learn/system-design/questions/slack) where hundreds of users need the same message? Or a live [Leaderboard](/learn/system-design/questions/leaderboard) viewed by thousands? This is where a **Pub/Sub message broker** (Redis Pub/Sub, Kafka, NATS) becomes essential. The flow works like this: 1. An event occurs (new chat message, score update, notification). 2. The event is published to a **topic** on the Pub/Sub broker (e.g., channel:general or leaderboard:weekly). 3. Every connection server that has clients subscribed to that topic receives the message. 4. Each connection server pushes the message to its locally connected, subscribed clients. This decouples the event producer from the delivery layer. The service that generates the event doesn't need to know which users are online or which servers they're connected to — it just publishes to a topic and the infrastructure handles the rest. **See it in practice:** The [Slack](/learn/system-design/questions/slack) question uses per-channel Pub/Sub topics for group messages and per-user topics for direct messages. The [Leaderboard](/learn/system-design/questions/leaderboard) question publishes rank changes to a leaderboard topic that all viewers subscribe to. The [Pub/Sub Messaging](/learn/system-design/design-patterns/pub-sub-messaging) pattern covers broker design in depth, and the [Fan-Out Feed Generation](/learn/system-design/design-patterns/fan-out-feed-generation) pattern extends this to social feed delivery. ## Adaptive Delivery Strategies Not all real-time updates need the same treatment. A direct message in [Slack](/learn/system-design/questions/slack) must arrive instantly and reliably, but a [Leaderboard](/learn/system-design/questions/leaderboard) viewed by millions can tolerate batching. Smart systems adapt their delivery strategy to the audience size. For **massive audiences**, pushing every update to every viewer is infeasible. Instead, you can serve periodic **CDN-based snapshots** of the current state (e.g., full leaderboard every 5 seconds) and supplement with **sampled live updates** (e.g., only push the top-10 changes in real time). This keeps the experience feeling live without overwhelming the connection infrastructure. ## Reconnection and Reliability Clients will inevitably disconnect — network switches, mobile sleep mode, brief outages, laptop lids closing. A robust real-time system handles this gracefully so users don't miss updates. **With SSE**, reconnection is largely handled for you. The browser's EventSource API automatically reconnects and sends Last-Event-ID back to the server, which can replay missed events. You just need to store recent events server-side (an in-memory ring buffer or Redis stream works well). **With WebSockets**, you must implement reconnection yourself: 1. Track a **sequence number** or **timestamp** per client on both sides. 2. When the client reconnects, it sends its last received sequence number. 3. The server replays all events since that sequence from a buffer or message log. 4. Use **exponential backoff** with jitter on reconnect attempts to avoid thundering herds of reconnecting clients after an outage. For critical data (financial transactions, game state), you may also want **acknowledgments** — the client confirms receipt of each message, and the server retries unacknowledged messages. This adds complexity but guarantees delivery. **See it in practice:** The [Slack](/learn/system-design/questions/slack) question implements message replay on reconnection using per-channel message sequences — when a user's connection drops and recovers, the server sends all messages they missed. The [Online Chess](/learn/system-design/questions/online-chess) question requires reliable delivery of game moves with acknowledgments since a missed move would desync the game state. The [Webhook](/learn/system-design/questions/webhook) question applies similar retry-with-backoff logic for outbound delivery, and the [Idempotency and Exactly-Once](/learn/system-design/design-patterns/idempotency-exactly-once) pattern covers how to ensure replayed messages aren't processed twice. ## Quick Decision Framework When you encounter a real-time requirement in a system design question, work through these questions in order: 1. **Does the client need to send data back frequently?** Yes: WebSocket. No: SSE (or even long polling for very low frequency). 2. **How many concurrent viewers?** Under 100K: direct Pub/Sub fan-out. Over 100K: add CDN snapshots and sampling. 3. **What happens if a message is missed?** Cosmetic issue (stale count): no replay needed. Functional breakage (missed chat message, desynced game): implement replay from sequence numbers. 4. **What's the update frequency?** Sub-second (games, collaboration): WebSocket with minimal framing. Seconds (feeds, dashboards): SSE works great. Minutes: polling is fine. The key insight is that **real-time doesn't mean one-size-fits-all**. The same application might use WebSockets for chat, SSE for notifications, and polling for analytics dashboards — each channel matched to its requirements.