aiohttp — Timeouts, Retries, and Pooling Patterns

Production-grade HTTP-client patterns for resilience and throughput.

Timeouts — every layer

timeout = aiohttp.ClientTimeout(
    total=30,           # entire request
    connect=5,          # DNS + TCP + TLS handshake
    sock_connect=5,     # individual socket connect
    sock_read=10,       # waiting for response chunks
)

What each one catches:

• total — the absolute ceiling. A 30s slow API call. A 30s slow read. A 30s slow TLS. Anything past this, the request is cancelled.
• connect — only the connection establishment phase. Useful for fast-failing unreachable peers: if you can’t connect in 5 seconds, give up and try the next replica.
• sock_read — between chunks of the response. A peer that started streaming but stalled mid-response triggers this. For non-streaming requests, total usually fires first.
• sock_connect — like connect but per-socket (matters for retries that try multiple addresses from a DNS lookup).

Rule: always set total. The 5-minute default will lose you sleep eventually.

Per-request override:

async with session.get(url, timeout=aiohttp.ClientTimeout(total=5)) as resp:
    ...

A long-running file upload may need total=600, but a regular API call should be total=5-10.

Retries — aiohttp has no built-in

Unlike httpx, aiohttp doesn’t ship with retries. You roll your own or use a library (aiohttp-retry).

Manual retry with exponential backoff + jitter

import asyncio, random

async def get_with_retry(session, url, attempts=3, base=0.1, cap=5.0):
    for i in range(attempts):
        try:
            async with session.get(url) as resp:
                if resp.status in (500, 502, 503, 504):
                    raise aiohttp.ClientResponseError(resp.request_info, resp.history, status=resp.status)
                resp.raise_for_status()
                return await resp.json()
        except (aiohttp.ClientConnectorError, aiohttp.ClientResponseError, asyncio.TimeoutError) as e:
            if i == attempts - 1:
                raise
            backoff = min(cap, base * 2 ** i) + random.random() * base
            await asyncio.sleep(backoff)

Key principles:

• Only retry idempotent ops — GET, PUT with idempotency key, POST with idempotency key. Never blind-retry POST.
• Retry transient errors only — connection errors, timeouts, 5xx. Never retry 4xx.
• Exponential backoff — base * 2 ** attempt with a cap.
• Jitter — random fraction of base. Prevents thundering-herd retries from a fleet.
• Cap attempts — 3-5 retries max. After that, fail loudly.

With aiohttp-retry

from aiohttp_retry import RetryClient, ExponentialRetry

retry_opts = ExponentialRetry(
    attempts=3,
    start_timeout=0.1,
    max_timeout=5.0,
    factor=2.0,
    statuses={500, 502, 503, 504},
)

session = aiohttp.ClientSession()
client = RetryClient(client_session=session, retry_options=retry_opts)

async with client.get(url) as resp:
    data = await resp.json()

Less boilerplate. Same idea.

Circuit breaker

When a dependency is dying, stop hammering it. Three states (closed → open → half-open). Libraries: aiobreaker, purgatory, or roll your own.

from aiobreaker import CircuitBreaker

breaker = CircuitBreaker(fail_max=5, reset_timeout=60)

@breaker
async def call_flaky_api(url):
    async with session.get(url) as resp:
        resp.raise_for_status()
        return await resp.json()

After 5 failures, the breaker opens — calls fast-fail without hitting the network. After 60s, half-open: one call attempts; success → closed; failure → re-open.

Combine with retries: per-call retry for transient blips; circuit breaker for sustained outages. Different timescales.

Connection pooling — sizing

The pool size limits concurrent requests. Two dimensions:

connector = aiohttp.TCPConnector(
    limit=100,              # total connections across all hosts
    limit_per_host=20,      # per host (default 0 = unlimited!)
)

How to size:

A FastAPI service handling 1000 concurrent requests, each making 1 outbound call → pool of 200 is too small (requests queue). 1000 is generous; tune via load testing.

limit_per_host=0 is the silent killer. Hits the same upstream with 1000 concurrent connections; the upstream OOMs or rate-limits. Always set a sane per-host limit (10-50 typical).

Semaphore — alternative concurrency bound

sem = asyncio.Semaphore(20)

async def fetch_one(url):
    async with sem:
        async with session.get(url) as resp:
            return await resp.text()

results = await asyncio.gather(*(fetch_one(u) for u in urls))

limit_per_host is a pool-level cap; Semaphore is an app-level cap. Use one or the other; using both unnecessarily complicates reasoning.

Keepalive

connector = aiohttp.TCPConnector(
    keepalive_timeout=30,           # close idle connections after 30s
    enable_cleanup_closed=True,
)

Default: 15s. For chatty service-to-service traffic, increase to reduce reconnect overhead.

Server-side: most reverse proxies / nginx default to 60-75s. Your client’s keepalive should be ≤ the server’s, otherwise you’ll hit “connection reset” errors when the server closes a connection your pool thinks is still alive.

Streaming responses

async with session.get(url) as resp:
    async for chunk in resp.content.iter_chunked(8192):
        process(chunk)

Don’t await resp.read() for large bodies — loads everything into memory. iter_chunked() streams.

For streaming uploads (file → API):

async def file_chunks(path):
    async with aiofiles.open(path, "rb") as f:
        while chunk := await f.read(8192):
            yield chunk

async with session.post(url, data=file_chunks("big.bin")) as resp:
    ...

Memory stays flat regardless of file size.

SSL configuration

import ssl

ssl_context = ssl.create_default_context()
ssl_context.load_verify_locations("/path/to/ca-cert.pem")

connector = aiohttp.TCPConnector(ssl=ssl_context)

For mutual TLS:

ssl_context.load_cert_chain("client.pem", "client.key")

ssl=False disables verification — never in production.

Authentication patterns

Bearer token

session = aiohttp.ClientSession(
    headers={"Authorization": f"Bearer {token}"}
)

Default headers apply to every request.

Per-request

async with session.get(url, headers={"Authorization": f"Bearer {token}"}) as resp:
    ...

Token refresh

class AuthClient:
    def __init__(self):
        self._session = None
        self._token = None
        self._expires_at = 0

    async def _refresh(self):
        async with self._session.post("/auth/token", ...) as resp:
            data = await resp.json()
            self._token = data["access_token"]
            self._expires_at = time.time() + data["expires_in"] - 30   # 30s buffer

    async def get(self, url):
        if time.time() >= self._expires_at:
            await self._refresh()
        async with self._session.get(url, headers={"Authorization": f"Bearer {self._token}"}) as resp:
            if resp.status == 401:
                await self._refresh()
                # Retry once
                async with self._session.get(url, headers={"Authorization": f"Bearer {self._token}"}) as resp2:
                    return await resp2.json()
            return await resp.json()

Refresh ahead of expiry; retry once on 401 (server’s view of expiry may differ).

Hooks / events

aiohttp doesn’t have a built-in hooks system as rich as httpx. For logging/metrics, wrap the session:

class InstrumentedClient:
    def __init__(self, session):
        self._session = session

    async def get(self, url, **kwargs):
        start = time.monotonic()
        try:
            async with self._session.get(url, **kwargs) as resp:
                latency = time.monotonic() - start
                metric.histogram("http_client_latency", latency, tags={"host": urlparse(url).host, "status": resp.status})
                return await resp.json()
        except Exception:
            metric.counter("http_client_error", tags={"host": urlparse(url).host})
            raise

Or use OpenTelemetry aiohttp instrumentation:

from opentelemetry.instrumentation.aiohttp_client import AioHttpClientInstrumentor
AioHttpClientInstrumentor().instrument()

Auto-injects trace context; spans for every outbound request.

Common pitfalls

• No total timeout — coroutines hang forever on bad peers.
• Per-request ClientSession — connection pool benefits gone.
• limit_per_host=0 (default) — saturate upstream / pool.
• Blind retry on POST — duplicate side effects. Idempotency keys required.
• Retry without backoff — thundering herd.
• ssl=False in production — MITM exposure.
• asyncio.gather over 10k coroutines — pool exhaustion, OOM. Bound with semaphore.
• await resp.read() on large responses — memory blow-up. Stream with iter_chunked.
• Mismatched keepalive timeouts (client > server) — sporadic “connection reset” errors.

Interview angle

• “How would you make HTTP calls reliably to a flaky API?” — bounded total timeout, exponential backoff with jitter on retries (only for idempotent ops), circuit breaker for sustained outages, connection-pool sizing for concurrency control.
• “limit_per_host=0 default — what’s wrong with that?” — under high concurrency to one host, you’ll saturate either the upstream or the pool with hundreds of concurrent connections. Set explicitly (e.g., 20).
• “How do you stream a large file upload through aiohttp?” — pass an async generator as data. aiohttp uploads chunks as the generator yields. Memory flat regardless of file size.
• “What does sock_read timeout catch?” — gaps between response chunks. A peer that started but stalled mid-response. For non-streaming, total usually fires first; for streaming responses, sock_read is the meaningful one.
• “Circuit breaker — when do you use it on top of retries?” — retries handle transient blips (one failed call); circuit breaker handles sustained outages (the dependency is dying — stop hammering it). Different timescales.
• “How do you handle token refresh in a long-running client?” — track expiry with a small buffer (30s); refresh proactively before expiry; on 401, refresh and retry once. Don’t loop retry-on-401 — could mask permanent auth failure.