Follow-Up Questions: Slow/Failing APIs, Auth, Retry, Tests

Short, interview-ready answers to common follow-ups after the system design or live-coding exercise.

1. What happens if one API is slow or fails?

If it’s slow:

• Timeouts: Every external call has connect and read timeouts (e.g. 5s connect, 30s read). A slow API doesn’t block the whole request forever; after the timeout we get an error and handle it.
• Concurrency: With async and asyncio.gather, the other APIs keep running. So if cars is slow, hotels and flights can still complete; we return what we have and optionally mark cars as failed or timed out (e.g. in SearchResult.errors).
• Circuit breaker: If one API is repeatedly slow or failing, the circuit opens after a threshold. We stop calling it for a cooldown period, fail fast for that source (or return fallback/cache), and the rest of the request can still succeed. That protects our app from one bad dependency.

If it fails:

• Retries: For transient failures (5xx, timeouts, connection errors), we retry a few times with exponential backoff (and jitter) in the client. We don’t retry 4xx (client error).
• Partial failure: We use return_exceptions=True in gather. If one API raises, we catch it, log it, add an entry to the response (e.g. errors: [{"source": "cars", "message": "..."}]), and return partial success (e.g. hotels and flights OK, cars empty). The user gets a 200 with the data we have and clear indication of what failed.
• Total failure: If all sources fail (or if we decide “any failure = fail the request”), we return 503 (or 502) with a clear message like “External services temporarily unavailable” and optionally queue a retry (e.g. background job) if that fits the product.

So: timeouts so one slow API doesn’t hang us; gather so others can still succeed; retries and circuit breaker for failures; partial vs total failure reflected in the response (200 with errors vs 503).

2. How do you handle authentication?

• With external APIs: We use whatever they require—usually API key or Bearer token in a header (e.g. X-API-Key, Authorization: Bearer <token>). We never put secrets in the URL. We load the key/token from environment variables (or a secret manager) and inject it into the client layer; the client adds the header to every request. We use different keys per environment (dev, staging, prod) and rotate them periodically.
• With our own API: If our service exposes an API, we use Bearer tokens (e.g. JWT) or API keys for machine clients. We validate the token in a dependency (e.g. FastAPI Depends(get_current_user)) and pass the authenticated identity to the service. We don’t put auth logic inside business logic; it stays at the router/dependency layer.
• Security: We never log or expose raw tokens or keys in errors or responses. We redact Authorization and similar headers in logs.

So: headers for credentials; env or secret manager for storage; client/dependency layer adds or validates auth; no secrets in code or URLs.

3. How do you retry safely?

• Only retry transient failures: We retry on 5xx, timeouts, and connection errors. We do not retry 4xx (bad request, unauthorised, not found)—same request would fail again.
• Idempotency: For non-GET calls (POST, PUT) we only retry if the operation is idempotent. We use idempotency keys when the external API supports them (same key = at most one side effect). Without that, we avoid retrying state-changing calls or we accept the risk and document it.
• Bounded retries: We cap the number of attempts (e.g. 3–5) and use exponential backoff (e.g. 1s, 2s, 4s) plus jitter so we don’t hammer the API and we avoid thundering herd with other clients.
• Where: Retry logic lives in the client (or a wrapper around it), so every call to that API gets the same policy. We don’t scatter retry logic across services.
• Circuit breaker: After many failures we open the circuit and stop calling for a cooldown. That avoids endless retries when the API is down and protects our resources.

So: retry only 5xx/timeouts, idempotency for state-changing calls, bounded backoff + jitter, centralised in the client, and circuit breaker to stop when the dependency is clearly down.

4. How do you structure tests?

• Service layer: We unit test the orchestration service by injecting mock clients. We set the mocks to return fixed data (e.g. list of cars, hotels, flights) and assert that the service returns the correct aggregated result (e.g. SearchResult with all three lists). We also test partial failure: one mock raises, others return data; we assert that the result contains the successful data and an entry in errors for the failed source. No real HTTP; tests are fast and deterministic.
• Clients: We unit test each client with a mock HTTP layer (e.g. httpx.MockTransport, or responses for requests). We stub a response (status, body) and assert that the client parses and maps it to our internal model correctly. We test error paths: 5xx or timeout → client raises our exception (e.g. ExternalApiError); 4xx → client raises a “client error” type. Optionally we test that retries happen (e.g. mock fails twice then succeeds; assert the transport was called 3 times) and that circuit breaker opens after N failures (next call doesn’t hit the transport).
• API / E2E: We test the router with the framework’s TestClient (e.g. FastAPI). We inject mock services or clients at the DI container so no real external calls. We assert HTTP status and response body for success and for partial failure (e.g. 200 with errors present when one source is mocked to fail). We don’t need to hit real APIs in the main test suite; we can add a few integration tests against sandbox APIs if available.
• Principles: Isolate with mocks; test behaviour (returned shape, errors, retries) not implementation details; one focus per test; no secrets in tests (use env or test doubles).

So: service tests with mock clients and partial-failure scenarios; client tests with mock HTTP and error/retry/breaker behaviour; API tests with TestClient and injected mocks; no real external calls in the core suite.

Interview angle

• “What is the theme behind all of these follow-ups?” - partial failure. Every question here is a variant of “one dependency misbehaves - what does the user see?” The strong answer names the timeout, the fallback, and what the response body says about the degraded part rather than claiming everything still works.
• “What do interviewers listen for?” - specific numbers and specific mechanisms: a connect and a read timeout with values, a retry policy that only covers idempotent calls, a circuit breaker with a threshold, and a bounded concurrency limit. “We would add retries” without a budget is the answer that invites the amplification follow-up.
• “What is the trap in the retry question?” - retrying a non-idempotent call, and retry amplification across layers - three layers each retrying three times is 27 requests at a service already failing. Idempotency keys and a single retry layer are the answers. See ../02_resilience/01_timeouts_retries_backoff.md.