REST Error Handling and HTTP Caching

Two cross-cutting REST concerns: consistent error responses (RFC 7807) and HTTP caching (ETag, Cache-Control). Both are underused.

Error responses — RFC 7807 Problem Details

The spec for “what should a JSON error response look like”:

HTTP/1.1 400 Bad Request
Content-Type: application/problem+json

{
  "type": "https://example.com/probs/validation-error",
  "title": "Validation failed",
  "status": 400,
  "detail": "Field 'email' must be a valid email address",
  "instance": "/users/42",
  "errors": [
    {"field": "email", "code": "invalid_format"},
    {"field": "age", "code": "out_of_range"}
  ]
}

Standard fields:

• type — URI identifying the problem class.
• title — short human-readable summary.
• status — HTTP status code (also in the response line).
• detail — human-readable explanation specific to this occurrence.
• instance — URI identifying this specific occurrence.

Plus extension fields (errors, correlation_id, etc.) per your needs.

Use Content-Type: application/problem+json — clients can switch on it to know it’s an error.

Why standardize?

Without a standard, every endpoint produces a different error shape:

{"error": "bad email"}
{"detail": "invalid email"}
{"message": "Email is invalid"}
{"errors": {"email": ["must be valid"]}}
{"code": "E001", "msg": "..."}

Frontend code becomes one giant if/else to extract the message. RFC 7807 fixes this.

Even without strict adherence, pick one error shape for your API and use it everywhere. The shape matters less than the consistency.

HTTP status codes

The set you’ll actually use:

See 03_status_codes.md for more.

422 vs 400

The eternal debate. Both mean “client’s fault.” Common convention:

• 400 Bad Request: the request is malformed (invalid JSON, missing required field, wrong types).
• 422 Unprocessable Entity: the request parses but fails business rules (email format wrong, age out of range, FK doesn’t exist).

DRF, FastAPI, and many frameworks use 422 for serializer validation failures. JSON:API uses 422. Other teams use 400 for everything client-side and never 422. Either’s defensible; be consistent.

401 vs 403

• 401: “I don’t know who you are.” No credentials or invalid credentials.
• 403: “I know who you are, you can’t do this.”

If your auth middleware returns 401 for anonymous and 403 for “wrong role,” that’s the standard. If you return 404 instead of 403 (to hide existence), it’s a security choice — explicit in API docs.

HTTP caching — the underused feature

HTTP has rich caching semantics most teams ignore. The headers:

ETag — strong validators

GET /users/42 HTTP/1.1
→
HTTP/1.1 200 OK
ETag: "v1-a8c2f3"
Cache-Control: private, max-age=60
{"id": 42, "name": "Alice"}

Client caches the response. Next time:

GET /users/42 HTTP/1.1
If-None-Match: "v1-a8c2f3"
→
HTTP/1.1 304 Not Modified

304 has no body. Saves bandwidth (especially helpful for big responses) and signals “your cached version is still good.” Server is still hit — that’s why ETag isn’t full caching, it’s revalidation.

ETag generation strategies:

• Hash of the response body: etag = hashlib.md5(json.dumps(data).encode()).hexdigest().
• Version column on the record: etag = f'"{user.id}-{user.version}"'.
• updated_at timestamp: etag = f'"{user.updated_at.timestamp()}"'.

Version-column or updated_at is cheaper than hashing the body.

Conditional updates (lost-update prevention)

PUT /users/42 HTTP/1.1
If-Match: "v1-a8c2f3"
→
HTTP/1.1 412 Precondition Failed   (if ETag has changed)
or
HTTP/1.1 200 OK + new ETag        (if matched)

If-Match makes updates conditional on the current version. Prevents the lost-update problem: A reads, B reads, A writes, B writes → B’s update silently overwrites A’s. With If-Match, B’s PUT fails with 412 — B’s client knows to refresh and merge.

This is optimistic locking at the HTTP layer. Useful for collaborative editing, distributed updates.

Cache-Control

Verbose; the directives that matter:

Examples:

# Public marketing page, cacheable everywhere for 5 min
Cache-Control: public, max-age=300

# User-specific data, browser-only, short cache
Cache-Control: private, max-age=60, must-revalidate

# Never cache (login pages, sensitive)
Cache-Control: no-store

# Immutable static asset (content-hashed filename)
Cache-Control: public, max-age=31536000, immutable

Vary — cache key dimensions

GET /api/items HTTP/1.1
Accept-Encoding: gzip
Authorization: Bearer ...
→
HTTP/1.1 200 OK
Cache-Control: public, max-age=300
Vary: Accept-Encoding, Authorization

Tells caches: “this response varies based on these request headers.” Without it, the CDN may serve the gzipped response to a client that didn’t request gzip, or one user’s data to another.

Vary: Cookie is dangerous — every user has a different cookie, so cache hit rate goes to ~0.

Cache-friendly API design

GET endpoints with stable URLs and ETags benefit from HTTP caching. POST/PUT/DELETE do not (always uncached).

If you have a hot read endpoint, design it as:

GET /api/articles/featured              # static URL
ETag: "v17"
Cache-Control: public, max-age=60, stale-while-revalidate=300

Add nginx proxy_cache or a CDN in front — backend hit goes from 10k RPS to ~17 RPS (one per minute, per cache key).

See ../12_protocols/nginx/08_caching.md.

When NOT to cache

• Per-user / authenticated data (unless Cache-Control: private).
• Frequently-updated data where staleness matters more than load.
• Anything with Set-Cookie (caches usually refuse).

Common pitfalls

• No ETag or Cache-Control — browsers heuristically cache, which is unpredictable. Set explicit headers.
• Caching auth-required endpoints publicly — Cache-Control: public on /api/me leaks one user’s data to another via CDN.
• Vary missing on gzipped responses — broken gzip-unaware clients.
• Conflating 401 and 403 — confuses clients.
• Different error shapes per endpoint — frontend has to handle each individually.
• POST returning 200 OK without a body — should be 201 Created (+ Location) for creates, 204 No Content for “done, nothing to return.”

Common interview confusions

• “PUT is for update, POST is for create.” — PUT is for “replace the resource at this URL with this representation” (idempotent). POST is for “do something at this collection URL” (not idempotent). Either can create or update depending on semantics.
• “422 doesn’t exist in HTTP/1.1.” — added by WebDAV (RFC 4918) and widely adopted for REST. Browser support is fine.
• “Caching breaks dynamic apps.” — short caches (60s) on hot read endpoints often shed 99% of the load without anyone noticing staleness.

Interview angle

• “How should error responses be structured in REST?” — RFC 7807 Problem Details: type, title, status, detail, instance + extensions like errors array. Content-Type: application/problem+json. Or pick a consistent custom shape and use it everywhere.
• “What’s the difference between 401 and 403?” — 401: not authenticated (no/invalid credentials). 403: authenticated but not authorized for this action.
• “400 vs 422?” — 400 for malformed requests (bad JSON, missing field). 422 for syntactically valid but semantically invalid (failed validation rules). Some teams use only 400.
• “What’s ETag for?” — version identifier for a resource. Client sends If-None-Match: <etag>; server returns 304 if unchanged, saving bandwidth. Also for If-Match on PUT to prevent lost updates.
• “How would you cache a public hot read endpoint?” — set Cache-Control: public, max-age=60, stale-while-revalidate=300 and a stable ETag. Put nginx or a CDN in front. Backend load drops from N requests/sec to ~1.
• “What does Vary: Accept-Encoding do?” — tells the cache to store different versions per Accept-Encoding header. Without it, a gzipped response can be served to a client that didn’t ask for gzip.
• “When should you NOT cache?” — per-user authenticated responses on public caches (use private), Set-Cookie responses, frequently-updated data where staleness matters.