Design ChatGPT (Streaming Chat with Branching & Share-link)
Problem statement
Design a streaming chat product like ChatGPT — token-by-token SSE rendering, the message tree that makes regenerate work, AbortController-based stop, and frozen share-link snapshots. Mentor-tone walkthrough with three flow diagrams.
Asked In
Key Challenges
- ·Token-by-token streaming that feels instant — the user must see characters appearing as the model generates them
- ·A conversation tree, not a flat list — every regenerate creates a sibling branch under the same parent message
- ·A Stop button that actually stops the model server, not just hides the UI
- ·Share-links that capture a frozen snapshot of one branch, viewable by anyone without the full chat history
- ·Resilience over flaky mobile networks — partial responses must be recoverable when the connection drops
2-Minute Answer — High-Level TL;DR
Read this before your interview
HLD Interview Focus
I would not design this as a flat chat list. ChatGPT is a conversation tree: every regenerate creates a sibling under the same parent, and a < 1/2 > picker switches which sibling is active — like checking out a git branch. Streaming rides SSE (HTTP that does not end): the client posts a message, keeps the response open, and appends token deltas into an assistant placeholder that was painted the instant the user hit Send. Stop must abort the fetch all the way through ReadableStream so the server stops burning tokens, not just hide a spinner. A share-link is a frozen snapshot of one path through that tree — a different route, read-only, CDN-cacheable, no composer. The hard parts are the tree shape, cancellation that reaches the server, and keeping Markdown/layout cheap while tokens arrive every few dozen milliseconds.
My Approach: Building the Solution
How to think about this problem
I'll start from what the user actually does — type, watch tokens appear, hit Stop, regenerate, share one answer — then draw the message tree on the whiteboard before any components. That data shape decides the URLs, the cache, and the share API. Only after the tree is clear do I pick SSE for one-way streaming and wire AbortController, branching, and share-links as layers on the same send lifecycle.
Why This Approach?
Most candidates design 'a chat app': a flat array of messages, a typing indicator, optimistic sends. That answer collapses the moment the interviewer asks what Regenerate does to the previous reply. ChatGPT does not overwrite — it keeps the old attempt as a sibling. Once you see the tree, regenerate, edit-and-re-run, branch pickers, and share-snapshots stop being UI chrome and become natural operations on parent_id + activeBranchByParent.
A ChatGPT thread is a git history of a thinking process. Sending a prompt is a commit. Regenerating is not `git commit --amend` — it is a sibling branch off the same parent. The < 1/2 > picker is checkout. A share-link is `git archive`: one frozen path, no ability to keep adding commits. Flat lists cannot express that; trees can.
Requirements: What Makes a Great Solution?
Clarify scope before designing anything
Requirements Exploration Questions
Ask your interviewer these questions to refine requirements
What is the scope of the chat product?
- ·Single-user conversations only, or shareable threads (multi-user view)?
- ·Persisted history across devices, or session-scoped?
- ·Multiple chats per user, or one rolling thread?
- ·Free tier with rate limits vs paid tier with higher limits — does the UI surface that?
Which features are in scope?
- ·Streaming text responses (always)
- ·Stop button to interrupt generation
- ·Regenerate to get a new response (creates a sibling branch)
- ·Edit user prompt and re-run (creates a new branch from that point)
- ·Share-link snapshots — read-only frozen views of one branch
- ·File attachments, voice mode, image generation — usually advanced/out of scope for the core design
What are the latency expectations?
- ·TTFT (time-to-first-token) target — typically <800ms p50, <2s p99
- ·Inter-token latency — visible smoothness depends on this; <80ms feels good
- ·Stop button responsiveness — should feel instant (<100ms)
- ·First page load — should render the chat shell with skeletons in <500ms
Which devices and networks?
- ·Mobile-first or desktop-first? Mobile keyboards, autoscroll-on-keyboard-open issues
- ·Network conditions: cellular dropouts, captive portals, browser tab throttling
- ·Tab-switching behavior: does the stream pause when tab is backgrounded? (Browsers throttle timers but not network)
What accessibility and i18n requirements?
- ·Screen-reader announcement of streaming responses (live regions)
- ·Keyboard-only operation: send, stop, regenerate, branch-switch
- ·RTL languages (Arabic, Hebrew) and bidi text in code blocks
- ·Multilingual responses including non-ASCII (Chinese, Japanese, emoji) — implies UTF-8 streaming care
Functional Requirements
Must Have — MVP
- ✓Send a prompt and receive a streamed response (token-by-token rendering)
- ✓Stop button that cancels generation server-side, not just hides UI
- ✓Conversation history within a single chat, persisted across reload
- ✓Regenerate any assistant message — creates a sibling branch
- ✓Branch picker UI (< 1/2 >) to switch between siblings
- ✓Markdown rendering for the assistant reply (headings, lists, code blocks with syntax highlighting)
- ✓Multi-chat sidebar with chat titles (auto-generated from first message)
- ✓Optimistic message echo — user prompt appears instantly when sent
Advanced (If Time Permits)
- +Edit user prompt and re-run (forks the tree at that point)
- +Share-link snapshots — POST /shares returns a public read-only URL
- +File attachments via presigned upload URL
- +Voice mode (separate WebSocket-based pipeline — see bonus)
- +Code-block 'copy' and 'run in playground' affordances
- +Per-message reactions / thumbs-up-down for eval feedback
Non-Functional Requirements
Performance Targets
- ·TTFT <800ms p50, <2s p99
- ·Inter-token latency <80ms (one render frame at 60fps is 16ms; render every 1–4 tokens to stay smooth)
- ·Initial page load: chat shell in <500ms, ready to type in <1s
- ·No layout thrash during streaming — bounded paint cost per token
Scale Targets
- ·Tens of thousands of concurrent streams per region
- ·Support chats with 1000+ messages without performance degradation (virtualization)
- ·Tens of millions of historical chats per region (storage is mostly cold)
Reliability & UX
- ·Network drop mid-stream: detect, show 'reconnecting', resume from last token via Last-Event-ID
- ·Tab-throttled mid-stream: stream pauses but resumes cleanly when tab refocuses
- ·Stop click never leaves the server generating orphaned tokens
- ·Regenerate failure rolls back to clean state with retry option
Accessibility
- ·Streaming response in role='log' aria-live='polite' so screen readers announce updates
- ·Keyboard shortcuts: Cmd/Ctrl+Enter to send, Esc to stop, Cmd/Ctrl+Shift+R to regenerate
- ·Focus management: after send, focus returns to composer; after stop, focus on Regenerate button
- ·Reduced motion: disable token fade-in animations under prefers-reduced-motion
Security & Privacy
- ·Auth on every API call; share-links are unguessable IDs (160-bit entropy minimum)
- ·Never expose internal prompt-system messages or model parameters in client
- ·Markdown rendering must be XSS-safe (sanitize HTML in code blocks; allowlist tags)
- ·Per-user rate limits enforced server-side (client UI shows quota but doesn't trust it)
Observability
- ·Track TTFT, inter-token latency, stream completion rate, abort rate
- ·Per-chat error rate dashboards (network drops vs server errors)
- ·Markdown render time profiling (top tail of long responses)
- ·Eval-feedback signals (thumbs-up/down) flow to model team
The full solution is part of HelloFrontend Pro
The question above is free to read in full. Upgrade to unlock the interactive workspace and the senior-level walkthrough that go with it.
- Full RADIO-framework walkthrough with architecture diagrams
- Data model, API design, and rendering strategy trade-offs
- Performance, accessibility, and offline sections plus a model answer
Already a member? Log in to open the workspace.