SDE2 design: 10K+ chat history
A detailed SDE2‑level system design thread breaks down handling power users with 10K+ chat messages — recommends reverse infinite scroll, cursor pagination to avoid offsets, and local optimistic caching for sub‑200ms loads. (x.com)
Field benchmarks show OFFSET-based APIs scanning millions of rows produced multi-second page loads in production, while switching to keyset/cursor-style paging returned page 1000 in ~45 ms during a real migration case study. (0x.run) A PostgreSQL experiment measuring pagination across a million rows reported a ~17× speedup when moving from offset scans to cursor/keyset approaches, and similar writeups cite even larger order-of-magnitude improvements at high page numbers. (milanjovanovic.tech) Client-side virtualization libraries built for messaging — React Virtuoso (active repo with chat-specific APIs) and its npm package (latest release in 2026) — provide imperative APIs for scroll anchoring and incremental insertion that keep DOM node counts low for 10k+ message histories. (github.com/petyosi/react-virtuoso) (npmjs.com/package/react-virtuoso) Production-ready optimistic-update patterns and local-cache strategies are documented with onMutate/rollback examples in TanStack/React Query, showing how to apply snapshot-and-rollback to keep UI latency under user-noticeable limits during network round-trips. (tanstack.com/query/latest/docs/framework/react/examples/optimistic-updates-cache) Operational stacks pairing an in-memory cache layer like Redis with async frameworks can push server P99 below tens of milliseconds (examples report sub‑50 ms P99 in feed/caching writeups), while HCI research sets 0.1–1.0 s as the canonical human response-time tiers (0.1 s feels instantaneous). (johal.in/fastapi-social-feeds-infinite-scroll-with-python-caching-2026-3/) (ekphorie.de/ham/pub/shows/1998gor/i_limits.html) System-design interviews at SDE2 typically expect concrete tradeoffs: justify keeping offset for small datasets (recommendations commonly cite a ~100k-row threshold), show a keyset SQL example, explain cache invalidation for hot chat windows, and name client libraries (Virtuoso) and cache patterns (TanStack optimistic updates) used to achieve tight latency SLAs. (0x.run) (github.com/petyosi/react-virtuoso)
