Case study /03 · Private build
AI Video Studio
Context
Text-to-video models made clips possible; they didn't make them directable. The products winning this category sell control — repeatable camera moves, consistent characters, predictable output — not raw model access. That framing drove every architecture decision.
Problem
Video diffusion is the most expensive inference workload in consumer AI: seconds of output cost minutes of GPU time. The platform had to keep queues honest under bursty demand, stream progress so waits feel intentional, stop unsafe generations before they burn GPU budget, and make motion control feel like a camera department, not a slot machine.
Constraints
- GPU capacity is the dominant cost: every architectural choice had to respect utilization.
- Generation latency is minutes, not milliseconds — the product experience had to be designed around honest asynchronous work.
- Moderation had to gate both inputs (prompts, reference images) and outputs before delivery.
- Model versions change fast; the pipeline had to swap checkpoints and motion modules without downtime.
Responsibilities
- Product design and the full web creation studio in Next.js.
- Python/FastAPI inference services wrapping video diffusion pipelines.
- GPU worker fleet orchestration: queueing, scheduling, retries and autoscaling signals.
- Motion-preset system encoding camera trajectories as reusable conditioning.
- Moderation pipeline across prompt, reference image and rendered output.
- Credits, plans and billing via Stripe; delivery via object storage and CDN.
Architecture
- The Next.js studio talks to a control-plane API; generation jobs enter Redis-backed queues partitioned by output tier so short previews never starve behind long renders.
- GPU workers pull jobs, run the diffusion pipeline (text/image conditioning, motion module, upscale and frame-interpolation stages) and checkpoint stage progress to Redis.
- Progress streams to the client over WebSockets — queue position, stage and frame preview thumbnails — so waiting is legible instead of a spinner.
- Motion presets are versioned data: named camera trajectories compiled into model conditioning, so the catalog can grow without code deploys.
- Moderation is a mandatory pipeline stage on both sides of generation; flagged jobs exit before expensive stages run.
- Finished renders land in object storage behind a CDN with signed URLs; PostgreSQL owns jobs, assets, credits and audit history.
Key decisions
- Preview-then-commit generation: cheap low-step previews first, full-quality renders only on user confirmation — the single biggest lever on GPU cost per satisfied output.
- Motion as presets rather than free-form prompting: constrained, named camera moves produce repeatable results users can build a visual language with.
- Queue fairness by tier and per-user concurrency caps instead of naive FIFO, keeping the free tier usable during spikes.
- Moderate before spending: input-side safety checks run on CPU before a job may touch a GPU.
- Credits ledger as an append-only table with idempotent operations — billing disputes are answerable from history, not guesswork.
Implementation
- Diffusion pipelines are staged (base generation → upscale → interpolation) with per-stage checkpoints, so a crashed worker resumes instead of restarting a render.
- Reference-image workflows support character consistency via LoRA-style conditioning attached at the pipeline level.
- The studio's timeline UI shows each job's stages with real thumbnails streamed as they decode.
- Model and motion-module versions are pinned per job for reproducibility; upgrades roll out queue-side with canary traffic.
- Webhook-driven Stripe integration reconciles subscriptions and one-off credit packs into the same ledger.
Performance
- Preview-first flow cut wasted full-quality renders substantially — most abandoned ideas die at the cheap stage.
- Batched scheduling keeps GPU utilization high across mixed job sizes without starving interactive previews.
- Stage checkpointing turned worker failures from full re-renders into partial resumes.
- CDN delivery with signed URLs keeps render download latency independent of inference infrastructure.
Outcomes
- A working end-to-end generative video platform: prompt or still in, motion-controlled clip out.
- The preset system delivers repeatable camera moves across generations — the product's core differentiator.
- The moderation-before-spend design held GPU costs to intended workloads.
- [PLACEHOLDER — add render volumes, latency numbers and user metrics when publishable.]
Lessons
- In GPU products, the pricing model and the architecture are the same design problem.
- Constrained creative controls (presets) beat maximal flexibility for user satisfaction and cost alike.
- Async product experiences live or die on progress legibility — streamed stages changed perceived wait more than actual speedups did.
Software Architecture
