Secrets management and rate limiting

Two often-bundled topics in interviews under “what does production-ready security look like.”

Secrets management

The hierarchy of bad to good:

Local dev — .env files

# .env (gitignored)
DATABASE_URL=postgres://localhost/mydb
API_KEY=sk_test_xxx

# python-dotenv loads it
from dotenv import load_dotenv
load_dotenv()

import os
db_url = os.environ["DATABASE_URL"]

Add .env to .gitignore immediately. Provide .env.example (committed) with placeholder values so others know what’s needed.

Production — env vars from a secret manager

# AWS Secrets Manager
import boto3, json
client = boto3.client("secretsmanager")
secret = json.loads(client.get_secret_value(SecretId="prod/api")["SecretString"])
db_password = secret["db_password"]

Don’t read secrets on every request — fetch at boot, cache, refresh on rotation. Use the platform’s automatic rotation if available.

For Kubernetes:

# Secret backed by AWS / GCP / Vault, mounted as env or file
env:
  - name: API_KEY
    valueFrom:
      secretKeyRef:
        name: app-secrets
        key: api_key

External Secrets Operator syncs from cloud secret manager → Kubernetes secrets.

Pre-commit guards

Block secrets at commit time so they never enter Git history.

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.5.0
    hooks:
      - id: detect-secrets

gitleaks is similar (Go-based, fast). git-secrets (AWS) targets AWS keys specifically.

If a secret is committed: rotate it (assume compromised), then scrub history (git filter-repo). Force-pushing rewrites is destructive — coordinate with the team.

Rotation

Every secret should have a defined rotation cadence. Cloud secret managers can rotate automatically (RDS passwords, IAM access keys). Application code must:

• Read fresh secret on a TTL or on receipt of a “rotated” signal.
• Tolerate brief overlap windows (old + new both valid for a few minutes).
• Not crash on a 401 from the dependency — refresh the secret and retry.

What never goes in logs

• Passwords.
• Full tokens (log a hash or last-4 if you need traceability).
• Full credit card numbers (PCI: only last 4 visible, even masked log lines must not contain the full number).
• API keys.

import structlog
log = structlog.get_logger()

# safe
log.info("auth_attempt", user_id=user.id, ip=request.client.host)

# never
log.info("auth_attempt", token=jwt_token, password=password)

Use a log-redaction layer for safety nets: regex over output, scrub anything matching common token patterns.

Rate limiting

The defenses against brute force, abuse, and runaway costs.

Algorithms

Token bucket is most common — 100 req/sec, burst 200 means steady-state 100 rps but a 200-request burst is fine.

Implementation — Redis

# Token bucket per user
import time, redis
r = redis.Redis()

def allow(user_id: str, rate: float = 10.0, capacity: int = 20) -> bool:
    key = f"rate:{user_id}"
    now = time.time()
    pipe = r.pipeline()
    pipe.hgetall(key)
    pipe.expire(key, 60)
    state, _ = pipe.execute()

    tokens = float(state.get(b"tokens", capacity))
    last = float(state.get(b"last", now))

    # refill
    tokens = min(capacity, tokens + (now - last) * rate)
    if tokens < 1:
        return False
    tokens -= 1

    r.hset(key, mapping={"tokens": tokens, "last": now})
    return True

For real systems, use a vetted library:

• slowapi (FastAPI/Starlette).
• django-ratelimit.
• aiolimiter (asyncio token bucket for outgoing calls).

from slowapi import Limiter
from slowapi.util import get_remote_address

limiter = Limiter(key_func=get_remote_address)
app.state.limiter = limiter

@app.get("/login")
@limiter.limit("5/minute")
def login(request: Request): ...

Where to limit

Different keys for different threats:

• Per-IP — coarse; defends against scraping and naive brute force.
• Per-user — for authenticated endpoints.
• Per-IP-per-user — stops account-sharing abuse.
• Per-token — when API keys correspond to plans/tiers.

Apply more strictly to expensive or sensitive endpoints: login (5/min), password reset (3/hour), forgot-password email (1/hour per address).

Returning 429

HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1716000060

Retry-After (seconds or HTTP date) is well-supported. The X-RateLimit-* headers are a de-facto convention.

Distributed rate limiting

Single-node counters don’t work across N app instances. Centralize the counter (Redis), or accept eventual-consistency:

• Redis-backed — the canonical option. Round-trip per request adds ~1ms.
• Local + sync — each node has a local counter, syncs occasionally; eventual but very fast.
• Edge — rate limit at CDN/WAF (Cloudflare, AWS WAF). Fastest, no app-side cost. Limited by what the edge can express.

Client-side handling

When you get 429:

import time, random

def call_with_retry(url, max_attempts=5):
    for attempt in range(max_attempts):
        resp = requests.get(url)
        if resp.status_code != 429:
            return resp
        retry_after = int(resp.headers.get("Retry-After", 1))
        time.sleep(retry_after + random.uniform(0, retry_after * 0.1))  # jitter
    raise RuntimeError("rate limited")

Always respect Retry-After. Add jitter so all your clients don’t retry simultaneously. Cap retries — at some point, fail loudly.

Login-specific defenses

• Progressive delay: 1st failure = 0s, 2nd = 1s, 3rd = 4s, 4th = 16s. User-experience friendlier than a hard lockout.
• Lockout after N: 10 failures in 10 min = locked for 30 min. Notify owner via email.
• CAPTCHA: after 3 failures, demand CAPTCHA before further attempts.
• Per-username + per-IP: attacker spreading across many usernames hits the IP limit.

Interview angle

• Q: “Where do you store API keys?” — env vars locally, secret manager (AWS Secrets Manager, Vault) in prod; never in source.
• Q: “What rate-limiting algorithms are common?” — fixed window, sliding, token bucket, leaky bucket; token bucket usually wins.
• Follow-up: “What do you put in the response when a client is rate-limited?” — 429 + Retry-After.
• Follow-up: “Distributed rate limiting — how?” — Redis-backed counter; or edge (CDN/WAF) for cheaper enforcement.

See 01_owasp_top_10.md, 04_secrets_config/.