System Designhard

Design a real-time event stream UI for high-volume telemetry

Design a frontend that displays a live stream of security events (think 5k-50k events/min) with filtering, virtualization, and pause/resume — without freezing the browser.

Asked at CrowdStrike, Netflix

#system design#real-time#virtualization#performance

Answer

Interview framing:

  • The hard parts are not the UI components; they are backpressure, batching, and memory.
  • A naive setState per WS message at 50k/min = main thread death.

Transport:

  • WebSocket (or SSE if one-way) with sequence numbers for gap detection.
  • Server-side filter/projection so the browser only receives subscribed event kinds.
  • Heartbeats every 15-30s; auto-reconnect with exponential backoff + resume-from-seq.

Ingestion pipeline (client):

  1. WebSocket onmessage pushes into a ring buffer (Array of fixed cap, e.g. 10k).
  2. A rAF/setTimeout-driven flusher (every ~100ms) drains the buffer into React state in one batched setState. This caps re-renders at ~10/s regardless of event rate.
  3. When buffer is full, drop oldest + increment a "dropped" counter shown in the UI (visible backpressure beats silent freeze).
  4. Pause button stops the flusher but keeps WS draining into the ring buffer; on resume, jump to the latest N or replay depending on UX choice.

Rendering:

  • Virtualized list (react-window / TanStack Virtual). Never render 10k DOM rows.
  • Row component memoized with stable keys (event.id), shallow-equal props.
  • Time-based grouping headers computed off-thread or memoized by minute bucket.

Filtering:

  • Apply server-side first (subscription filter).
  • Client-side secondary filter runs on the buffered slice, not the raw stream.
  • For complex queries, debounce the filter input and run the predicate in a Web Worker so the main thread stays at 60fps.

Observability of the UI itself:

  • Track dropped count, lag (now - latest event ts), reconnect count, render time.
  • Expose them in a debug panel so SREs can diagnose "feed feels slow" reports.

Pitfalls to call out:

  • React state with append-only arrays grows unbounded — cap it.
  • Date.now() formatting per row per render is surprisingly hot — memoize or use Intl.RelativeTimeFormat sparingly.
  • Auto-scroll to bottom only when user is already at bottom; otherwise users lose their place.

Practise more System Design questions →