Distributed Counters

Problem Statement

Design a backend system that powers YouTube long-form video view counters.

Key user behaviors (what the system should provide)

1. Watch a video → the system records viewing activity and eventually increments the video's view count.
2. Refresh / reconnect / retry (mobile drops, browser refresh, client retries) → the system avoids double-counting and still converges to the right number.
3. Watch a popular (viral) video → the view count remains available and updates smoothly even under massive traffic spikes.

Out of Scope

• Shorts view counting rules, ads view measurement, monetization/billing counters
• Creator analytics beyond the public counter (unique viewers, watch time, retention curves)
• Recommendation/search ranking and user personalization

Functional Requirements

FR1 — Ingest view events

Accept view signals for a video_id and persist them so the system can later decide whether a view should be counted.

FR2 — Serve view counts (watch page)

Provide GetViewCount(video_id) to return the public view count quickly and reliably for end users.

Non-Functional Requirements

NFR1 — Correctness

• No double-counting on retries: the same logical view must be counted at most once (target: 0 duplicate increments per idempotency/dedup key).
• No lost updates under concurrency: when many writes hit the same video_id (including the same shard), increments must not overwrite each other.

NFR2 — Latency / Freshness

• GetViewCount(video_id) latency: p95 ≤ 50 ms, p99 ≤ 150 ms
• View count freshness (staleness of displayed number): p95 ≤ 2 s, p99 ≤ 5 s

NFR3 — Scale / Hotspot Resistance

• Peak ingest capacity: ≥ 200K view-events/sec (example target)
• Hot video tolerance: ≥ 50K increments/sec for a single video_id without a single partition/shard melting
• Read scale: support very high QPS for GetViewCount via caching/edge serving

NFR4 — Availability > Consistency

• For view counts we choose AP: always accept view events and always serve a count, even during partitions, and we rely on event-log + rollups to converge to the correct number.

Challenge Yourself — Why AP is the Right CAP Choice Here?

• Partition tolerance is non-negotiable. In a global system, cross-region partitions and partial network failures will happen.
• When a partition happens, you don't want the watch page to break. Returning some view count is far better than timing out or erroring.
• For views, the product can tolerate temporary staleness. Users don't require the number to be globally exact to the second.

So under partition:

• Reads stay available (serve last known count).
• Writes stay available (continue accepting view events regionally).
• Counts converge later once replication/rollup catches up.

What we "give up" (and how we contain it)?

• We give up strong consistency (no global linearizable counter).
• We contain the impact by:
  • using an event log as the source of truth,
  • doing regional ingestion + aggregation,
  • and rolling up behind a watermark so the number converges predictably.

Requirement Summary