Scaling WebSocket Applications

WebSocket scaling is harder than HTTP scaling. Connections are long-lived and stateful, pinned to one server. Naive load balancing puts user A on server 1 and user B on server 2 — they can’t talk without coordination. Solutions exist; all involve a shared broker.

For HTTP-side scaling see ../../28_networking/10_load_balancers.md.

The fundamental problem

Alice  → [LB] → [server 1] (Alice connected here)
Bob    → [LB] → [server 2] (Bob connected here)

Alice sends a message intended for Bob.
Server 1 doesn't know Bob exists.

Three solutions, in order of complexity:

1. Stick all WebSockets to one server — only works at tiny scale.
2. Sticky sessions + per-room sharding — keep related connections on one server.
3. Shared broker (Redis pub/sub, NATS, Kafka) — servers coordinate via a backend; the dominant pattern.

Sticky sessions

The load balancer routes each WebSocket connection to a specific backend based on some identifier and keeps it there for the duration.

upstream backend {
    ip_hash;            # route by client IP
    server backend1:8000;
    server backend2:8000;
}

Sticky sessions are required for WebSockets in many setups. But:

• Sticky alone doesn’t solve the cross-server messaging problem.
• Sticky balances “first connect” but doesn’t rebalance after.
• Server failure breaks all its sticky sessions; clients reconnect to other servers but lose state.

The Redis pub/sub fan-out pattern

The dominant production pattern:

                    ┌─────────────┐
       ┌──── Alice ──┤  Server 1   │──┐
   [LB]│             └─────────────┘  │
       │                                ↓ pub
       │             ┌─────────────┐    [ Redis pub/sub ]
       └──── Bob ────┤  Server 2   │  ↑      ↑
                     └─────────────┘  │ sub  │ sub
                                      │      │
                                  Server 1   Server 2
                                  (delivers to Alice)  (delivers to Bob)

Steps:

1. Each server subscribes to relevant channels in Redis (e.g., per-room: room:abc123).
2. When a server receives a message, it publishes to the Redis channel.
3. Redis distributes to all subscribed servers.
4. Each server delivers to its locally-connected clients.

# Simplified Redis pub/sub broker
import redis.asyncio as redis

class Broker:
    def __init__(self):
        self.redis = redis.from_url("redis://localhost")
        self.pubsub = self.redis.pubsub()
        self.local_subscribers = {}    # channel → set of WebSockets

    async def subscribe(self, channel, websocket):
        if channel not in self.local_subscribers:
            await self.pubsub.subscribe(channel)
            self.local_subscribers[channel] = set()
        self.local_subscribers[channel].add(websocket)

    async def unsubscribe(self, channel, websocket):
        self.local_subscribers[channel].discard(websocket)
        if not self.local_subscribers[channel]:
            await self.pubsub.unsubscribe(channel)
            del self.local_subscribers[channel]

    async def publish(self, channel, message):
        await self.redis.publish(channel, message)

    async def run_dispatcher(self):
        async for msg in self.pubsub.listen():
            if msg["type"] == "message":
                channel = msg["channel"].decode()
                content = msg["data"].decode()
                for ws in self.local_subscribers.get(channel, set()):
                    try:
                        await ws.send_text(content)
                    except Exception:
                        pass    # connection dead; cleanup elsewhere

run_dispatcher is a long-running task; one per server.

For Django Channels: this is what the channel_layer (channels-redis) does for you out of the box.

Channel naming patterns

The shape of your Redis channels reflects your access pattern:

A server with N connections subscribes to up to N channels. Hot rooms (with 10k subscribers) span many servers; each server has one subscription to the channel and demuxes locally.

NATS, Kafka, RabbitMQ as alternatives

Default: Redis pub/sub. If you need durability (replay messages to reconnecting clients), Redis Streams or Kafka.

WebSocket apps typically don’t need Kafka-level durability — messages are ephemeral live events. If the client missed it, they probably get it from the database on reconnect.

Horizontal scaling — adding more servers

              [LB with sticky sessions]
             ↙        ↓        ↓        ↘
     [Server 1] [Server 2] [Server 3] [Server 4]
            ↘         ↓         ↓         ↙
              [ Redis pub/sub broker ]
                          ↕
                    [PostgreSQL]
                    (state of record)

To add capacity: add another server. It subscribes to relevant channels; the LB routes new connections to it (sticky-session aware).

Capacity per server depends on:

• File descriptors (ulimit -n).
• Per-connection memory.
• Message processing CPU.

Typical: 10k–100k concurrent connections per server. Past that, more servers.

For 1M+ concurrent: dedicated WebSocket tier separate from API tier, with optimized server processes (often Go, Rust, or Erlang for performance).

Sharding by entity

For massive scale, single-Redis pub/sub becomes a bottleneck. Shard:

hash(room_id) % N → Redis cluster node

Channel "room:abc123" lives on Redis node 0
Channel "room:xyz456" lives on Redis node 1

Redis Cluster handles this transparently. Or app-level sharding to multiple separate Redis instances.

Same principle for the WebSocket tier:

hash(room_id) % N → assigned WebSocket server pool

All members of room abc123 connect to pool 0
All members of room xyz456 connect to pool 1

Now within a pool, even Redis pub/sub isn’t needed for that room (everyone is on the same servers). Across pools is rare since rooms don’t span pools.

Trade-off: rebalancing is complex. Adding a pool requires re-hashing rooms, which means disconnecting and reconnecting users.

Load balancing — L4 vs L7

For WebSockets, L7 is usually preferred:

• Can route /ws/chat to chat servers, /ws/notifications to notification servers.
• Cookie-based stickiness more accurate than IP-based (NAT’d users all share an IP).
• WebSocket handshake is HTTP/1.1; L7 inherently understands the upgrade.

L4 is simpler and faster but lacks the routing flexibility.

Connection migration (rare)

Some advanced setups can move a WebSocket between backend servers without the client noticing. Examples:

• gRPC-style proxies (Envoy) can in some cases transparently reconnect to a different backend.
• In-process pinning + Redis: WebSocket terminates at a thin frontend; actual logic runs server-side; logic can move servers, the WebSocket stays.

Most production apps don’t bother. Accepting that “server failure = client reconnects” is simpler.

Health checks for WebSocket servers

LBs need to know which backends are healthy:

# Kubernetes liveness / readiness for an ASGI app
livenessProbe:
  httpGet:
    path: /health
    port: 8000
readinessProbe:
  httpGet:
    path: /ready
    port: 8000

/health: process alive. /ready: ready to accept new connections (not draining, all downstream deps OK).

When deploying, /ready should return failing during connection draining. The LB stops sending new connections; existing ones finish gracefully.

Graceful shutdown for rolling deploys

shutdown_event = asyncio.Event()

async def shutdown_handler():
    # signal start of drain
    shutdown_event.set()
    # close all active WebSockets
    for conn in active_connections:
        try:
            await conn.close(code=1001, reason="Server shutting down")
        except Exception:
            pass

signal.signal(signal.SIGTERM, lambda *_: asyncio.create_task(shutdown_handler()))

Workflow:

1. SIGTERM arrives (orchestrator notifying impending shutdown).
2. /ready starts returning failing → LB stops routing new connections.
3. Wait for in-flight messages to finish.
4. Send close frames (code 1001) to all connections.
5. Clients reconnect to other servers.
6. Process exits.

Kubernetes’ terminationGracePeriodSeconds defaults to 30s. Tune for your message turnaround.

Connection budget per server

Rough capacity planning:

Assuming ~10-50 KB per connection (app state + buffers). Numbers go up if connections are mostly idle, down if message rate is high.

Bottlenecks (in order of likelihood):

1. CPU on message processing (if hot).
2. Memory per connection.
3. File descriptors (raise ulimit -n).
4. Network bandwidth (rare for chat-like loads; common for video).

Cost reality

For 100k concurrent connections:

• 4-10 servers (depending on size).
• 1 Redis instance (or small cluster).
• Load balancer.
• Bandwidth.

Bare-metal / VPS cost: ~$200-500/month. Cloud (with HA, redundancy): $1500-5000/month.

For 10M concurrent (Twitch, Discord scale):

• Hundreds of servers.
• Optimized non-Python stack (Go, Rust, Erlang/Elixir, Bun).
• Custom routing layer, not off-the-shelf.

Python WebSocket scales to “large medium” workloads. For true massive scale, switch language or architecture.

Common pitfalls

• No broker → cross-server messages don’t work — users can connect, can’t broadcast.
• In-memory connection list assumed to be global — only correct in single-process mode.
• No sticky sessions — repeated reconnects bounce across servers; per-connection state lost each time.
• Redis as a SPOF — single Redis fails, all WebSocket coordination breaks. Use Redis Sentinel or Cluster.
• Subscribing to Redis channels per connection — 100k connections × N channels = 100k subscribers. Better: one subscription per (server, channel) regardless of local subscriber count.
• No graceful shutdown — every deploy kicks all users; UX suffers.
• CPU-bound message processing on a single event loop — async doesn’t parallelize CPU. Multiple worker processes share the load.

Common interview confusions

• “Sticky sessions solve scaling.” — only the routing problem. You still need a broker for cross-server messaging.
• “WebSocket can’t horizontally scale.” — it can; just needs a broker. Redis pub/sub is the standard pattern.
• “Python is too slow for WebSocket.” — Python handles 10k-100k connections per process comfortably. For 1M+, switch tools.

Interview angle

• “How do you scale WebSockets across multiple servers?” — sticky sessions to keep individual connections pinned to one backend, plus a shared pub/sub broker (Redis is standard) so servers can coordinate cross-server messaging. Each server subscribes to relevant channels; publishes go through Redis; each server delivers to its local connections.
• “Why does WebSocket need sticky sessions?” — connection is stateful, pinned to one server. Naive round-robin LB would create a new connection on each request (which doesn’t apply here since the connection persists) or break the existing one. Sticky session keeps the connection on one server for its lifetime.
• “How does broadcasting to a chat room work across servers?” — one or more users in the room are connected to each server. When a message arrives at server A, it publishes to Redis channel room:abc. All servers (B, C, …) subscribed to that channel receive the message and deliver to their locally-connected users in the room.
• “What’s the role of Redis pub/sub in WebSocket scaling?” — message bus between servers. Servers publish events; other servers receive and forward to their local connections. Without it, a message arriving at server A only reaches users on server A.
• “What happens when a WebSocket server is terminated?” — graceful shutdown: stop accepting new connections (readiness probe fails), send close frame 1001 (“Going Away”) to existing connections, clients reconnect via LB to another server, missed messages handled via your resume protocol if any.
• “How many WebSocket connections can one server handle?” — typically 10k-100k per Python process. Limited by file descriptors, per-connection memory (~10-50 KB), CPU for message processing. Beyond that, more processes / servers.
• “L4 or L7 load balancer for WebSockets?” — L7 (ALB, nginx, Envoy) is usually preferred — understands HTTP upgrade, allows path-based routing, cookie-based stickiness more accurate than IP-based. L4 (NLB, HAProxy TCP) is simpler / faster but less flexible.
• “What’s an alternative to Redis pub/sub for WebSocket coordination?” — NATS (similar pattern, faster), Kafka (durable, can replay), Redis Streams (durability + replay), RabbitMQ (routing flexibility). Redis pub/sub is the default; switch when you need durability or higher throughput.