Streaming isn't magic. It's just an HTTP response that doesn't end immediately. The server keeps the connection open and writes bytes to it as it has them. Your fetch call resolves with a Response object whose body is a ReadableStream — and you read from that stream chunk by chunk until the server says it's done.

Most LLM APIs put a specific format on top of this called Server-Sent Events (SSE). Each "frame" is a data: <json> line followed by a blank line. Your job: read bytes, decode them to characters, split on the blank-line delimiter, parse each frame as JSON, and update your UI.

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  const frames = buffer.split('\n\n');
  buffer = frames.pop() ?? '';  // last is partial — save for next iteration
  for (const frame of frames) {
    const data = frame.replace(/^data: /, '');
    if (data === '[DONE]') return;
    onDelta(JSON.parse(data).delta);
  }
}

That nine-line loop is the heart of every streaming chat in the world. Memorize it.

Streaming UIs need a Stop button, and the Stop button has to actually stop. The wrong way is to set a cancelled boolean in state and check it inside your loop. State updates close over the value at function creation, so by iteration ten your loop is reading a stale snapshot. The user thinks they cancelled; the stream keeps going.

The right primitive is AbortController. Create one, pass controller.signal to fetch, and call controller.abort() to cancel. The pending fetch rejects with an AbortError, the connection closes, your loop exits.

const ctrl = new AbortController();
const res = await fetch('/api/chat', { signal: ctrl.signal });
// later, when user hits Stop:
ctrl.abort();

Here's a bug that only shows up after you launch internationally. You read a chunk of bytes, convert to a string, render it. Works perfectly in your tests. But every now and then a UTF-8 character — an emoji, a Chinese character, a Hindi vowel — splits across two chunks. Your decode produces a question-mark replacement character. The user sees garbage. You can't reproduce it locally.

The fix is one parameter: always use TextDecoder with { stream: true }. That flag tells the decoder to hold back any partial multibyte sequence until the next chunk arrives.

const decoder = new TextDecoder();
buffer += decoder.decode(value, { stream: true });
//                                ^^^^^^^^^^^^ this prevents the bug

Without the flag, the decoder finalizes whatever bytes you give it, even if they're incomplete. With it, you get correct text every time. Cost is zero. Add it as muscle memory.

For most LLM APIs this isn't a problem — tokens arrive at maybe 30 per second and React handles that comfortably. But if you connect to a fast inference endpoint, or render multiple streams concurrently, you can hit a regime where setState-per-token drops frames. The typing indicator stutters, scroll feels sticky, things get janky.

The mitigation is to batch token writes onto requestAnimationFrame. Accumulate incoming chunks in a ref, flush to state once per frame. Your effective update rate caps at 60fps no matter how fast tokens arrive.

const pending = useRef('');
const rafId = useRef(null);

const onChunk = (chunk) => {
  pending.current += chunk;
  if (rafId.current) return;
  rafId.current = requestAnimationFrame(() => {
    setMessage((m) => m + pending.current);
    pending.current = '';
    rafId.current = null;
  });
};

One thing — don't add this on day one. Measure first. The fastest way to look junior in a senior interview is to optimize before you have a problem.

Things to plan for, in rough order of how often they actually hurt:

• Network blip mid-stream — the connection drops, you have half a message. Mark it as interrupted in the UI (italic, a small badge) instead of silently leaving partial text. Offer "Retry from here."
• Backgrounded tabs — browsers throttle timers when a tab loses focus. Your stream may stall and resume. Listen for visibilitychange and show a subtle indicator that the response is paused.
• Memory growth — at 50KB per message and 200 messages, you're at 10MB of React-tracked strings. Virtualize the message list (react-virtuoso) once you cross ~50 turns.
• Screen readers — without aria-live="polite" on the assistant message wrapper, blind users get no announcement of the streamed reply.
• Race on rapid sends — user hits Stop, sends again before cleanup runs. Tag each stream with an id; only apply chunks if their id matches the latest. Otherwise late chunks from the cancelled stream append to the new message.