System Designhard
Design a real-time event stream UI for high-volume telemetry
Design a frontend that displays a live stream of security events (think 5k-50k events/min) with filtering, virtualization, and pause/resume — without freezing the browser.
Asked at CrowdStrike, Netflix
Answer
Interview framing:
- The hard parts are not the UI components; they are backpressure, batching, and memory.
- A naive setState per WS message at 50k/min = main thread death.
Transport:
- WebSocket (or SSE if one-way) with sequence numbers for gap detection.
- Server-side filter/projection so the browser only receives subscribed event kinds.
- Heartbeats every 15-30s; auto-reconnect with exponential backoff + resume-from-seq.
Ingestion pipeline (client):
- WebSocket onmessage pushes into a ring buffer (Array of fixed cap, e.g. 10k).
- A rAF/setTimeout-driven flusher (every ~100ms) drains the buffer into React state in one batched setState. This caps re-renders at ~10/s regardless of event rate.
- When buffer is full, drop oldest + increment a "dropped" counter shown in the UI (visible backpressure beats silent freeze).
- Pause button stops the flusher but keeps WS draining into the ring buffer; on resume, jump to the latest N or replay depending on UX choice.
Rendering:
- Virtualized list (react-window / TanStack Virtual). Never render 10k DOM rows.
- Row component memoized with stable keys (event.id), shallow-equal props.
- Time-based grouping headers computed off-thread or memoized by minute bucket.
Filtering:
- Apply server-side first (subscription filter).
- Client-side secondary filter runs on the buffered slice, not the raw stream.
- For complex queries, debounce the filter input and run the predicate in a Web Worker so the main thread stays at 60fps.
Observability of the UI itself:
- Track dropped count, lag (now - latest event ts), reconnect count, render time.
- Expose them in a debug panel so SREs can diagnose "feed feels slow" reports.
Pitfalls to call out:
- React state with append-only arrays grows unbounded — cap it.
- Date.now() formatting per row per render is surprisingly hot — memoize or use Intl.RelativeTimeFormat sparingly.
- Auto-scroll to bottom only when user is already at bottom; otherwise users lose their place.