WebSocket Connection Management

The “connection stays open” property of WebSocket isn’t free. Idle connections die behind firewalls. Slow clients block writes. Networks drop. Browsers reload. The state in between connection open and close needs careful design.

Heartbeats — keeping the connection alive

Network intermediaries (NATs, load balancers, firewalls, corporate proxies) drop “idle” TCP connections. Common timeouts:

A WebSocket with no traffic for 60s + behind an ALB = silently dropped TCP. The next send fails with broken pipe.

Solution: heartbeats. Either side sends periodic frames; if a response doesn’t come back in time, declare the connection dead.

Ping / pong frames (protocol-level)

The WebSocket spec defines ping (opcode 0x9) and pong (opcode 0xA) control frames.

# websockets library — automatic
async with serve(handler, "0.0.0.0", 8765,
                 ping_interval=20,   # send ping every 20s
                 ping_timeout=20):    # close if no pong within 20s
    ...

The library handles ping/pong transparently; your handler code doesn’t see them. Connection auto-closes if pongs stop arriving.

App-level heartbeats (alternative)

setInterval(() => ws.send(JSON.stringify({type: "ping"})), 20000);

ws.onmessage = (e) => {
  const msg = JSON.parse(e.data);
  if (msg.type === "pong") return;
  // ... handle other messages
};

# Server
if msg.get("type") == "ping":
    await websocket.send_json({"type": "pong"})

App-level heartbeats also work; useful when:

• You don’t trust the library’s ping implementation.
• You need finer control (different intervals per client tier).
• You want the ping to also carry information (server timestamp, queue depth).

Cost: a few bytes every 20-30s per connection. For 10k connections, ~50 KB/s of pure ping traffic. Negligible.

Timeouts to configure

# nginx
location /ws {
  proxy_pass http://backend;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";

  proxy_read_timeout 3600s;       # 1 hour, longer than heartbeat interval
  proxy_send_timeout 3600s;
}

Set proxy_read_timeout longer than your heartbeat interval. Otherwise nginx drops the connection between heartbeats.

For ALB: configure idle timeout > heartbeat interval.

Reconnection — client side

Connections WILL drop. Clients must reconnect.

let ws;
let backoff = 1000;
const MAX_BACKOFF = 30000;

function connect() {
  ws = new WebSocket("wss://example.com/ws");

  ws.onopen = () => {
    console.log("connected");
    backoff = 1000;     // reset on successful connection
  };

  ws.onclose = (e) => {
    console.log(`closed (${e.code}): ${e.reason}`);
    if (e.code === 4001) {
      // auth failure — don't reconnect
      return;
    }
    setTimeout(connect, backoff);
    backoff = Math.min(backoff * 2, MAX_BACKOFF);  // exponential backoff
  };

  ws.onerror = (e) => console.error(e);
}

connect();

Key elements:

• Exponential backoff: 1s, 2s, 4s, 8s, … cap at 30-60s.
• Don’t reconnect on terminal errors: 4001 (auth failed), 4003 (banned), etc. Application-defined.
• Reset backoff on successful connection: not just on open, but on a successful message exchange. (A connection that opens but closes immediately is “successful” too easily.)
• Jitter: add random 0-1s to the delay so thousands of clients reconnecting after a server restart don’t synchronize and thunder.

const delay = backoff + Math.random() * 1000;
setTimeout(connect, delay);

Resume after reconnect

If your protocol has state (you missed messages while disconnected), you need a resume mechanism:

ws.onopen = () => {
  ws.send(JSON.stringify({
    type: "resume",
    last_event_id: lastSeenEventId,
  }));
};

Server:

@app.websocket("/ws")
async def ws(websocket: WebSocket):
    await websocket.accept()
    initial = await websocket.receive_json()
    if initial.get("type") == "resume":
        missed = await fetch_events_since(user_id, initial["last_event_id"])
        for event in missed:
            await websocket.send_json(event)
    # ... normal handling

Three resume strategies:

The strategy depends on your application:

• Chat: per-event ID (don’t miss messages).
• Live dashboard: snapshot + delta (state is the source of truth; missed deltas don’t matter if you have current state).
• Game: at-most-once (state is high-rate; the next tick fixes things).

Backpressure — slow clients

A slow client (bad network, lots of messages) can’t read as fast as the server writes:

Server sends rapidly → kernel send buffer fills → next send() blocks

In async code, the blocking happens at await websocket.send(...). While the server awaits, it can’t serve other work for this connection.

Strategies:

Per-connection send queue with bound

class Connection:
    def __init__(self, ws):
        self.ws = ws
        self.queue = asyncio.Queue(maxsize=100)
        self.task = asyncio.create_task(self.sender())

    async def send(self, msg):
        try:
            self.queue.put_nowait(msg)
        except asyncio.QueueFull:
            # Slow client; either drop or disconnect
            await self.ws.close(code=1011, reason="Slow consumer")

    async def sender(self):
        while True:
            msg = await self.queue.get()
            await self.ws.send(msg)

Each connection has a bounded outgoing queue. Producer (broadcaster) puts; sender (task) takes. If the queue fills (slow client), drop or kick.

Bounded queue prevents one slow client from consuming unbounded memory.

Drop policy

For lossy applications:

async def send(self, msg):
    try:
        self.queue.put_nowait(msg)
    except asyncio.QueueFull:
        # Drop oldest, push new
        _ = self.queue.get_nowait()
        self.queue.put_nowait(msg)

Newest data wins. Good for live data (stock prices, game state) where stale messages are useless.

Direct kick

async def send(self, msg):
    try:
        self.queue.put_nowait(msg)
    except asyncio.QueueFull:
        await self.ws.close(code=1011)
        raise ConnectionAborted

For situations where falling behind is unacceptable (real-time trading, monitoring), just close. Client reconnects + resumes.

Per-message size limits

# websockets library
async with serve(handler, max_size=2**20, max_queue=32):     # 1 MB max message, 32 message queue
    ...

# FastAPI doesn't expose this directly; configure via Uvicorn:
# uvicorn ... --ws-max-size 1048576

Without limits, a malicious client sends a 10 GB message → server OOMs.

Set both:

• Max message size: per the spec (close code 1009 if exceeded).
• Max queue depth: per connection; prevents one connection from buffering unbounded data.

Cleanup on disconnect

Always release per-connection state when a WS closes:

@app.websocket("/ws")
async def ws(websocket: WebSocket):
    await websocket.accept()
    user_id = authenticate(websocket)
    register_connection(user_id, websocket)
    try:
        while True:
            msg = await websocket.receive_json()
            await handle_message(user_id, msg)
    except WebSocketDisconnect:
        pass
    except Exception as e:
        logger.exception("WS error")
    finally:
        unregister_connection(user_id, websocket)
        # cancel any per-connection tasks, free buffers, etc.

Common leaks:

• Connection still in your “active connections” list after close → broadcasts try to send to dead sockets.
• Subscriptions on Redis pub/sub not unsubscribed.
• Per-connection asyncio tasks still running.

Use try/finally or context managers to make cleanup unmissable.

Graceful shutdown

When the server shuts down (deploy, restart), in-flight WebSockets need to close cleanly:

@app.on_event("shutdown")
async def shutdown():
    for conn in active_connections:
        try:
            await conn.close(code=1001, reason="Server shutting down")
        except Exception:
            pass

Code 1001 (Going Away) tells clients to reconnect after a brief delay. With proper client backoff, deploys don’t cause user-visible disruption.

For rolling deploys: drain connections from one instance before terminating. Modern orchestrators (Kubernetes) signal SIGTERM and wait terminationGracePeriodSeconds (default 30s) before SIGKILL. Use that window to send close frames.

Connection limits

# OS file descriptor limit
ulimit -n 65535

# nginx
events { worker_connections 65535; }

# Application-level
MAX_CONNECTIONS = 50000
if len(active_connections) >= MAX_CONNECTIONS:
    await websocket.close(code=1013, reason="Server at capacity")

Each WebSocket consumes:

• 1 FD on the server.
• ~10-50 KB of kernel buffers per connection.
• ~10-100 KB of application state per connection (depending on app).
• 1 FD per upstream connection (if proxying).

10k connections ≈ 200 MB minimum. 100k ≈ 2-20 GB. Plan accordingly.

Common pitfalls

• No heartbeats → connections die silently behind firewalls.
• No client reconnect → page refresh = new connection storm; bad UX otherwise.
• No backoff on reconnect → clients hammer the server during outages.
• Synchronized reconnect storms → server restart, 10k clients reconnect at the exact same instant. Jitter the backoff.
• Unbounded send queues → slow client exhausts memory.
• Forgetting cleanup → leaked subscriptions, ghost connections in broadcast list.
• No max message size → DoS via giant payloads.
• Heartbeat interval > proxy timeout → connection drops between heartbeats.

Common interview confusions

• “WebSockets handle disconnects automatically.” — the spec defines close frames; reconnection is the application’s responsibility.
• “Ping/pong is for measuring latency.” — primarily for connection liveness. Latency measurement is a side use.
• “TCP keepalive is enough.” — TCP keepalive defaults are wrong (often 2 hours). Application-level heartbeats every 20-30s are required for real-world infra.

Interview angle

• “How do you keep a WebSocket connection alive?” — heartbeats. Protocol-level ping/pong frames (handled by the library; configure ping_interval and ping_timeout) or app-level {type: "ping"} JSON messages. Interval shorter than infrastructure idle timeouts (typically 20-30s vs 60s LB timeout).
• “How do you implement reconnection on the client?” — exponential backoff (1s → 2s → 4s → … capped at 30-60s) plus jitter. Reset backoff on successful connection. Don’t reconnect on terminal close codes (auth failure). Resume protocol if missed messages matter (track last_event_id).
• “What’s backpressure in WebSocket?” — a slow client can’t read as fast as the server writes; the send buffer fills, await send() blocks. Mitigations: per-connection bounded queue with drop-oldest or close-on-full policy.
• “How do you handle graceful shutdown?” — on SIGTERM, send close frame (code 1001 “Going Away”) to each active connection. Client backoff reconnects to the new instance. Pair with orchestrator’s terminationGracePeriodSeconds.
• “Why do you need application-level heartbeats?” — TCP keepalive defaults are too long (2 hours typical); intermediate infrastructure (load balancers, firewalls) drops idle connections after 30-600s. Heartbeats every 20-30s keep them alive and detect dead peers.
• “What’s the resume pattern after reconnect?” — client tracks last seen message ID; on reconnect, sends {type: "resume", last_event_id: X}; server replays missed events. Alternative: snapshot + delta (server pushes current state).
• “How many WebSocket connections can one server handle?” — typically 10k-100k per process, limited by file descriptors, memory per connection (~50 KB-100 KB), and CPU for message processing. Raise ulimit -n and worker_connections in nginx. For more, horizontal scale + sticky LB.