When writing a client for external APIs, what should you consider (auth, logging, credentials storage, retries, exponential backoff)?

Answer

Auth

• Store secrets in env vars or a secret manager, never in code or in repo.
• Use the auth mechanism the API expects (API key header, OAuth2 client credentials, etc.) and centralize it in the client (e.g. one httpx client with auth= or a custom transport).

Logging

• Log requests/responses at a configurable level (e.g. DEBUG): method, URL, status, duration, correlation ID. Redact headers/fields that may contain secrets or PII.
• Log failures and retries so production issues are traceable.

Credentials storage

• No hardcoding. Use os.environ, .env (loaded only in dev), or a vault. Prefer short-lived tokens and refresh them inside the client when possible.

Retries and exponential backoff

• Retry only on transient conditions: network errors, 429 (rate limit), 5xx. Do not retry 4xx (except maybe 429) without a clear strategy.
• Use exponential backoff: wait 1s, then 2s, then 4s, etc., with a cap and optional jitter to avoid thundering herd.
• Set a max retry count and/or total timeout so one failing dependency doesn’t hang the app.
• In Python, httpx with custom transport or tenacity (or similar) can implement this; many clients also support a retries option.

Other

• Timeouts: always set connect and read timeouts.
• Circuit breaker (optional): after repeated failures, stop calling the API for a period to fail fast and allow recovery.
• Idempotency: for mutating calls, consider idempotency keys if the API supports them.

Interview angle

• “How do you structure an outbound API client?” - one class per provider owning base URL, auth, timeouts, retries and error mapping, constructed once at startup and injected. Business code calls typed methods and never sees HTTP.
• “Why reuse a single AsyncClient?” - connection pooling and TLS session reuse. Creating a client per request discards both and adds a handshake to every call, which is a measurable and very common performance bug.
• “Where does the retry policy live?” - in the client, once, not scattered across service methods. Otherwise you can’t reason about total latency or retry amplification. See ../../../system_design/02_resilience/01_timeouts_retries_backoff.md.
• “How do you test it?” - respx against recorded real payloads for the client itself, and a fake implementing the same Protocol for service-level tests.