← System Design

Stream Over HTTP First

mediumCore ConceptAI
Asked at
GitHub
OpenAI
Perplexity
Stripe
Vercel
GitHub, OpenAI, Perplexity, Stripe, Vercel

Problem statement

SSE is HTTP that does not end. Stream LLM tokens and one-way events before reaching for WebSockets — with Stop, buffering, and a clear when-not decision.

Asked In

OpenAIAnthropicVercelGitHubPerplexityStripe

Key Challenges

  • ·Streaming tokens or events without inventing a WebSocket
  • ·Choosing SSE vs fetch+ReadableStream vs WebSockets
  • ·Stopping mid-stream cleanly (AbortController / EventSource.close)
  • ·Parsing chunks that split mid-event or mid-JSON
★

2-Minute Answer — High-Level TL;DR

Read this before your interview

HLD Interview Focus

For one-way server push over HTTP I start with SSE or fetch plus a ReadableStream. SSE gives EventSource auto-reconnect and a simple data: protocol. Modern AI SDKs often use fetch streaming so they can POST and send auth headers. I reach for WebSockets only when the client must send continuously or we need binary frames. I always wire Stop with AbortController, and I buffer chunks until a full SSE event or NDJSON line is complete.

→

My Approach: Building the Solution

How to think about this problem

User hits Send in a chat. Tokens should appear as the model thinks — not after a 12-second spinner. That product moment is the streaming decision.

Why This Approach?

Weak answers open with WebSockets by default. Strong answers pick the thinnest HTTP stream that fits, name Stop and reconnect, and know when WebSockets actually win.

SSE is a radio broadcast: the station talks, you listen, you can change the dial (close). WebSockets are a phone call: both sides talk, and you own the whole call setup.

R

Requirements: What Makes a Great Solution?

Clarify scope before designing anything

Requirements Exploration Questions

Ask your interviewer these questions to refine requirements

Is the stream one-way or two-way?

  • ·Client asks once, server streams
  • ·Both sides send continuously
  • ·Binary audio/video?

What must Stop and reconnect do?

  • ·Cancel in-flight generation
  • ·Resume with Last-Event-ID
  • ·Show partial answer safely

Functional Requirements

Must Have — MVP

  • ✓Typed stream events on the client
  • ✓Stop / abort path
  • ✓Buffer across chunk boundaries
  • ✓Clear SSE vs WebSocket decision

Advanced (If Time Permits)

  • +Last-Event-ID resume
  • +HTTP/2 multiplexing for many streams
  • +BFF that hides provider keys

Non-Functional Requirements

Interview bar

  • ·Name the invariant in one sentence
  • ·Give one concrete failure of the naive approach
  • ·State when the pattern is the wrong tool
Premium

The full solution is part of HelloFrontend Pro

The question above is free to read in full. Upgrade to unlock the interactive workspace and the senior-level walkthrough that go with it.

  • Full RADIO-framework walkthrough with architecture diagrams
  • Data model, API design, and rendering strategy trade-offs
  • Performance, accessibility, and offline sections plus a model answer
Unlock the full solution →

Already a member? Log in to open the workspace.

More System Design questions

View all System Design →