frontend / frontend system design / 02_infinite_feed.md

Design: Infinite Feed (Twitter-Style)

5 min read source

Design: Infinite Feed (Twitter-Style)

TL;DR

A vertically scrolling list that loads more as you reach the end. The interesting parts are cursor pagination (offset breaks under writes), virtualization (rendering 10K DOM nodes is fatal), scroll restoration (back-button), new-content insertion (the “12 new posts” banner), and media optimization (most rows have images/video). The senior answer is “list of decisions,” not “I’d use useInfiniteQuery.”

Requirements to clarify

  • Item count ceiling. 1K? 100K? Unbounded? Drives virtualization and unload strategy.
  • Insertion model. Append-only? Real-time push from the top? Read-position memory?
  • Media. Image-heavy, video-heavy, text-only? Drives bandwidth budget and lazy loading.
  • Item height. Fixed (easier) or variable (needs dynamic measurement)?
  • Multi-column / responsive. Single column on mobile, multi-column at desktop widths?
  • Offline / partial offline. Cached pages survive reload?

API contract

Cursor pagination only — offset breaks the moment writes happen.

GET /api/feed?after=<cursor>&limit=20
→ 200 {
    "items": [{ "id": "p123", "createdAt": "...", "author": {...}, "body": "...", "media": [...] }],
    "nextCursor": "cur_abc",
    "newerCursor": "cur_xyz"           // for "fetch newer than this" polls
  }

Server returns:

  • nextCursor (continue scrolling down)
  • newerCursor (cursor anchor for “newer than” polling — for the “12 new posts” banner)

See ../11_apis_data_fetching/04_pagination_and_infinite_queries.md for the cursor design and useInfiniteQuery recipe.

Client data model

  • TanStack useInfiniteQuery stores data.pages[] — each page is a server response. Use select to flatten for rendering.
  • Cursor for “new posts” banner kept in a separate useQuery polling every 30s with the saved newerCursor.
  • Scroll position mirrored to URL or sessionStorage for restoration.

Virtualization — non-negotiable above ~200 items

DOM rendering scales linearly; 10K nodes destroys scroll perf and memory. Use @tanstack/react-virtual (or react-window):

const parentRef = useRef<HTMLDivElement>(null);
const virtualizer = useVirtualizer({
  count: items.length,
  getScrollElement: () => parentRef.current,
  estimateSize: () => 120,         // average row height
  overscan: 5,                     // render 5 above/below viewport
  measureElement: (el) => el.getBoundingClientRect().height,  // for variable height
});

return (
  <div ref={parentRef} style={{ height: 600, overflow: "auto" }}>
    <div style={{ height: virtualizer.getTotalSize(), position: "relative" }}>
      {virtualizer.getVirtualItems().map(v => (
        <div
          key={v.key}
          ref={virtualizer.measureElement}
          data-index={v.index}
          style={{ position: "absolute", top: 0, transform: `translateY(${v.start}px)` }}
        >
          <FeedItem item={items[v.index]} />
        </div>
      ))}
    </div>
  </div>
);

Variable-height items need measureElement so the virtualizer caches measured heights and adjusts subsequent positions. Without that, scroll jumps as heights are discovered.

Fetching trigger — IntersectionObserver sentinel

<div ref={sentinelRef} />   // last element in list
useEffect(() => {
  const obs = new IntersectionObserver(([e]) => {
    if (e.isIntersecting && hasNextPage && !isFetchingNextPage) fetchNextPage();
  }, { rootMargin: "400px" });
  if (sentinelRef.current) obs.observe(sentinelRef.current);
  return () => obs.disconnect();
}, [hasNextPage, isFetchingNextPage]);

rootMargin: "400px" fires the fetch before the sentinel actually enters the viewport — the user shouldn’t see a loading spinner unless the network is genuinely slow.

Scroll restoration

User scrolls 200 items deep, clicks a post, hits back. Native behavior loses position.

Strategies:

  • Persist scroll offset + loaded-cursor list in sessionStorage keyed by route URL. On mount, re-prefetch all pages up to where the user was, then jump to the saved scroll offset.
  • Next.js App Router automatically restores scroll for <Link> navigation if the layout cache still has the pages — works if the data is in the router cache.
  • Cheat: scroll restoration only works if heights are deterministic — virtualized lists need cached measurements to restore precisely.

This is the #1 polish item users notice. Production feeds (Twitter, Reddit) lose scroll position constantly and people complain.

“New posts at the top” insertion

Polling every N seconds with newerCursor returns a count of new posts (not the posts themselves — they may be many).

UX pattern:

  • A subtle banner appears: “12 new posts — click to load.”
  • Clicking prepends them to the list only if the user is at the top — otherwise inserting at index 0 jumps their scroll position. If they’re scrolled down, the banner stays until they scroll up, then it inserts.
const { data: newCount } = useQuery({
  queryKey: ["feed", "newer", newerCursor],
  queryFn: () => fetch(`/api/feed/count?after=${newerCursor}`).then(r => r.json()),
  refetchInterval: 30_000,
});

{newCount > 0 && <Banner onClick={prepend}>{newCount} new posts</Banner>}

For Twitter-volume firehose, replace polling with SSE/WebSocket — see 04_chat.md for the connection pattern.

Media optimization

  • loading="lazy" on <img> for native lazy loading (modern browsers).
  • <picture> with srcset + sizes — serve the right resolution per device pixel ratio and viewport width.
  • AVIF / WebP with JPEG fallback via <picture> <source>.
  • Aspect ratio in CSS to reserve space (aspect-ratio: 16 / 9) — prevents CLS when images load.
  • Video poster + lazy embed — don’t load the video tag until in viewport.
  • Decode hints: decoding="async" lets the browser decode off-thread.

A feed with 30 unoptimized images can ship 50MB; with srcset + AVIF it’s 2-3MB.

Failure modes

  • Stale data from cache while user expects new — pair staleTime with manual refresh affordance (“pull to refresh” mobile, header click web).
  • Mutation creates an item not yet in cache (user posts) — optimistically prepend to page 0; on server confirmation, replace temp id with real.
  • Item disappears between fetches (deleted) — TanStack Query refetch reconciles; the row vanishes on next refresh, or you handle 410 Gone in detail-fetch.
  • Slow page load mid-scroll — show a small inline loader in the sentinel; never block scroll.
  • Network drop — TanStack retries; if all retries fail, surface a banner.
  • Backgrounded tab — pause polling; resume on focus.

Telemetry

  • p99 fetch latency per page.
  • Sentinel-trigger to first-paint of new page time (catches slow render even if network is fast).
  • Scroll-to-bottom abandonment.
  • LCP per page (first-screen content).
  • INP for scroll responsiveness.

What a senior is expected to say

  • “Cursor pagination — offset breaks under writes and is slow at depth. Server returns nextCursor (older) and newerCursor (anchor for polling new content).”
  • “Virtualize above ~200 items. Variable heights need measureElement or you get jumpy scroll. Render overscan: 5 so scroll feels native.”
  • “IntersectionObserver with rootMargin: 400px triggers the next fetch before the user sees a spinner.”
  • “New posts go behind a banner — never insert into a scrolled-down list without consent, scroll jump is awful.”
  • “Media is the perf killer — loading=lazy, srcset/sizes, AVIF/WebP, aspect-ratio to prevent CLS.”
  • “Scroll restoration is the polish item. Persist cursors + scroll offset in sessionStorage keyed by URL.”

Cross-references

Further reading

Previous
Design: Typeahead / Autocomplete