Design Sora

Requirements#

Functional Requirements#

We'll scope to four FRs. Each one maps to exactly one flow in the High Level Design.

FR1 — Generate

A user submits a text prompt (optionally with an image or video as conditioning input) and generation parameters (duration up to 20s, resolution up to 1080p, aspect ratio) to create a video.

FR2 — Track

A user can see the status and progress of their generation, including a low-resolution preview before the final render completes.

FR3 — View & browse

A user can watch their finished videos and browse a public explore feed of videos shared by others.

FR4 — Remix

A user can take an existing video (theirs or a public one) and re-generate it with a modified prompt, preserving lineage back to the original.

Out of scope (say this out loud): model training and fine-tuning pipelines, billing/credits, mobile capture, editing timelines (storyboards), and recommendation ranking for the feed beyond a simple recency/popularity ordering. Each of these is its own interview.

Non-Functional Requirements#

NFR1 — Latency is asymmetric. Submission acknowledgment must be fast (< 500ms p99), but generation itself is minutes-scale and users accept that — if we give honest progress and a preview. Target: draft-quality preview within 30–60s; final 1080p render within 2–10 min depending on tier.

NFR2 — Cost efficiency over raw latency. GPU fleet utilization is the dominant cost line. We optimize for high utilization (target > 60–70%) and fair queueing over minimizing any single job's wait.

NFR3 — Availability with graceful degradation. The control plane (submit, status, view) targets 99.9%. The GPU data plane can degrade — under capacity crunch, queue times grow and free-tier resolution drops, but submissions never fail outright.

NFR4 — Durability. Generated videos are user-created content; once a generation succeeds, the asset must not be lost (11 nines object-store durability).

NFR5 — Safety is a hard gate, not best-effort. No video reaches the user or the public feed without passing prompt-level and frame-level moderation. This is a correctness requirement, not an optimization.

NFR6 — Provenance. Every output is watermarked and carries C2PA metadata identifying it as AI-generated.

Scale & Back-of-Envelope Estimations#

Assume a successful consumer launch:

The GPU fleet math — the number that identifies the real system:

Daily demand = 5M generations × 3 GPU-min = 15M GPU-min/day
             = 250K GPU-hours/day
             → ~10,400 GPUs running continuously
At 60% achievable utilization → provision ~17K H100s

This single number justifies the entire scheduling deep dive: a 5% utilization improvement is ~850 GPUs, i.e., tens of millions of dollars a year. The API tier, by contrast, is trivially small — a handful of stateless servers.

Interviewer signal: candidates who compute the GPU fleet size have identified the real system; candidates who dwell on API QPS have not.

Follow-up Questions to Expect#

• [DD1] Scheduling: How do you keep ~17K GPUs highly utilized with jobs of wildly different sizes and three user tiers?
• [DD2] Inference: What happens inside a generation, and how does that shape what the scheduler can do (preemption, batching, checkpointing)?
• [DD3] Safety: Where do the moderation gates sit? Fail open or fail closed when classifiers are down?
• [DD4] Storage & delivery: How do you store 55PB/year and serve 1.5PB/day of egress without melting the origin?

Core Thinking & Design Pattern#

Before drawing boxes, orient around three tensions that shape everything:

1. The queue is the product#

With minutes-long jobs and a fixed GPU fleet, you cannot buy your way out of a spike in real time (GPU capacity has procurement lead times of months). So the system's job is to shape demand: priority tiers (paid vs. free), fair scheduling per user, preview-first generation so users get value early, and honest progress so waiting feels acceptable. This is closer to designing a build farm or a render farm than a CRUD service.

2. Separate the control plane from the data plane#

Submissions, status reads, and feed browsing are cheap, high-QPS, stateless-friendly operations that should stay up even when the GPU fleet is saturated or partially down. Generation is expensive, stateful, and failure-prone. Different availability targets, different scaling models, different teams. The boundary between them is a durable queue.

3. Safety and provenance are in the critical path#

Unlike a photo-upload product where moderation can be async-after-publish, a generative system creates the risky content itself. The moderation gates (prompt-time and frame-time) sit inline before delivery, which means their latency and availability budgets are part of the generation SLO.