backend / architecture design / 18_idempotency_keys.md

Idempotency Keys (API-Level Deep Dive)

7 interview angles 6 min read source

Idempotency Keys (API-Level Deep Dive)

A client retries a POST that may have succeeded. Without idempotency, you charge their card twice. With idempotency keys, the second request returns the first response and does nothing else.

Pattern (Stripe-style)

Client generates a unique key per logical operation:

POST /v1/charges
Idempotency-Key: a3f7b9d1-1c8c-4f8c-9e6e-1234abcd5678
Content-Type: application/json

{ "amount": 1000, "currency": "usd", "customer": "cus_42" }

Server stores (key → response). Same key replayed → return stored response unchanged.

HTTP/1.1 201 Created
Idempotency-Replayed: true
{ "id": "ch_5", "amount": 1000, ... }

Data model

CREATE TABLE idempotency_keys (
    key            TEXT PRIMARY KEY,
    user_id        BIGINT NOT NULL,
    request_hash   TEXT NOT NULL,
    method         TEXT NOT NULL,
    path           TEXT NOT NULL,
    status         TEXT NOT NULL,             -- 'in_progress', 'completed', 'failed'
    response_code  INT,
    response_body  JSONB,
    locked_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
    completed_at   TIMESTAMPTZ,
    expires_at     TIMESTAMPTZ NOT NULL DEFAULT now() + INTERVAL '24 hours'
);

CREATE INDEX idx_idempotency_expires ON idempotency_keys (expires_at);

Scope the key to user_id + method + path. Two different users with the same key by coincidence shouldn’t collide; the same user replaying the same key should hit the same record.

Server flow

@app.post("/v1/charges")
async def charge(req: ChargeIn, idempotency_key: str = Header(...), user = Depends(current_user)):
    request_hash = hashlib.sha256(json.dumps(req.dict(), sort_keys=True).encode()).hexdigest()

    async with db.transaction():
        record = await db.fetchrow("""
            SELECT * FROM idempotency_keys
            WHERE key = $1 AND user_id = $2
            FOR UPDATE
        """, idempotency_key, user.id)

        if record:
            if record["request_hash"] != request_hash:
                raise HTTPException(422, "Idempotency-Key reused with different payload")
            if record["status"] == "completed":
                return JSONResponse(record["response_body"], status_code=record["response_code"])
            if record["status"] == "in_progress":
                # Someone else is currently processing this key
                raise HTTPException(409, "Conflict: request in progress")

        # First time: create row in-progress
        await db.execute("""
            INSERT INTO idempotency_keys (key, user_id, request_hash, method, path, status)
            VALUES ($1, $2, $3, $4, $5, 'in_progress')
        """, idempotency_key, user.id, request_hash, "POST", "/v1/charges")

    # Outside the lock: do the work
    try:
        charge_obj = await stripe.charge(req)
        body = charge_obj.dict()

        await db.execute("""
            UPDATE idempotency_keys
            SET status = 'completed', response_code = 201, response_body = $1, completed_at = now()
            WHERE key = $2
        """, body, idempotency_key)

        return JSONResponse(body, status_code=201)
    except Exception as e:
        await db.execute("""
            UPDATE idempotency_keys
            SET status = 'failed', completed_at = now()
            WHERE key = $1
        """, idempotency_key)
        raise

The tricky middle states

Request mid-flight: client retries before first response arrived

A network blip — client gives up on the original request and retries with the same key. Server sees status='in_progress'. Options:

  • Reject with 409 (simplest). Client backs off and retries; eventually the original completes and the second retry will return the stored response.
  • Block briefly waiting for in-progress to complete (timeout, then 409). Better UX but more complex.

Stripe’s actual behavior: returns 409 Conflict on concurrent retries with the same key.

The original request crashed mid-processing

Row is in_progress. The original Python process died. The retry sees in_progress and rejects. What now?

Solutions:

  • Time-bound in_progress. If locked_at < now() - 30 seconds, treat as failed and let the retry proceed.
  • Background sweeper that flips stale in_progress to failed.

Without this, a single crash leaves the idempotency key stuck until manually cleared.

Same key, different payload

if record["request_hash"] != request_hash:
    raise HTTPException(422, "Idempotency-Key reused with different payload")

Don’t silently return the stored response — that masks a client bug. 422 forces the client to either correct the payload or use a new key.

Storage and retention

Idempotency keys live for 24h typically (Stripe’s window). After that, deletion is safe — a request that old is genuinely a new request.

DELETE FROM idempotency_keys WHERE expires_at < now();

Run as a periodic job. Without retention, the table grows forever.

Scope: what counts as “the same request”

The request hash should cover:

  • Request body.
  • URL path parameters.
  • Significant query parameters.
  • Potentially the user’s authorization context (user_id).

Should NOT cover:

  • Timestamp headers (Date:).
  • Tracing IDs (X-Trace-Id:).
  • Volatile metadata.

Hash a normalized, sorted JSON representation.

Idempotency keys + outbox + at-least-once = exactly-once

Idempotency at the API layer + outbox at the side-effect layer + at-least-once delivery downstream = the system processes each logical request exactly once, end-to-end.

Client: POST /charges with Idempotency-Key: K
Server: (check K) → already done? Return stored.
                → not done? Process, store result keyed by K.
        Inside the same transaction, write outbox event "ChargeSucceeded".
Relay:  reads outbox, publishes to Kafka.
Consumer: dedupes by event ID.

Robust to: client retry, server crash, broker outage, consumer crash, network blips. Each layer handles its own redelivery; the keys/IDs tie everything together.

Distributed concerns

If your API runs on N pods, all see the same idempotency_keys table — the DB row lock (FOR UPDATE) serializes concurrent retries of the same key. The lock is per-row; doesn’t affect requests with different keys.

For ultra-high-volume APIs where DB lock contention matters, use a Redis-based fast path: check Redis first, fall through to DB on miss.

Client responsibilities

  • Generate UUID per logical operation, persist client-side. Same logical op → same key. New logical op → new key. Bad clients reuse keys for unrelated operations and get 422s.
  • Retry on network errors, NOT on 4xx. A 422 means the request is bad; retrying doesn’t help.
  • Reasonable retry backoff. Exponential with jitter.

Stripe’s SDK does this automatically; many DIY clients don’t.

Common bugs

  • Caching by key without checking request hash. Client sends {amount: 100}, then with same key sends {amount: 100000}. Server returns the 100 response, refuses the 100000 charge — but client thinks the 100000 succeeded.
  • No retention. Table grows forever; eventually slows down lookups.
  • In-progress lock never expires. Stuck keys after crashes.
  • Storing the key without scoping to user. Cross-user collision (unlikely with UUIDs but real with short keys).
  • Returning 200 instead of the stored status code. Stored response should preserve original status.
  • Idempotency on GET requests. GETs are idempotent by HTTP design. The idempotency-key pattern is for POST/PUT/DELETE.

Interview angle

  • “How do you handle a client retrying a POST charge after a network timeout?” — idempotency keys. Client generates a UUID per logical operation, sends as Idempotency-Key header. Server stores (key → response); same key replayed returns the stored response, doesn’t re-execute. Stripe-style.
  • “What if the client retries with the same key but a different payload?” — return 422 (or 409). Indicates a client bug — same logical operation should mean same payload. Don’t silently return the original response; that hides the bug.
  • “How long do you keep idempotency keys?” — 24 hours is the typical window (Stripe’s default). Old enough that legitimate retries have already happened; short enough to bound storage. Periodic cleanup job.
  • “What’s the race condition with concurrent retries of the same key?” — both requests arrive almost simultaneously. Without a lock, both proceed and you charge twice. Solution: row-level lock (SELECT … FOR UPDATE) on the idempotency_keys row before processing. The second request waits or 409s.
  • “What if the original request’s server crashes mid-processing?” — row stays in_progress until either a sweeper marks it failed or locked_at ages out. Without that, retries see in_progress and 409 forever. Time-bound the in-progress state.
  • “Why scope keys by user_id?” — defense in depth against accidental cross-user key collisions and abuse. UUIDs make collisions astronomically unlikely, but scoping is cheap insurance.
  • “How do idempotency keys compose with outbox?” — API layer: idempotency keys ensure the work runs once. Outbox layer: the resulting event publishes reliably (at-least-once). Consumer layer: dedupes by event ID. Together: exactly-once processing end-to-end.
Previous
Rate-Limiting Algorithms