setdefault always evaluates the default

The gotcha

d.setdefault(key, default) — the default is a regular function argument, evaluated every time, even when the key already exists. If default is expensive (a DB call, a network request, a fresh allocation), you pay the cost on every call.

Minimal repro

def expensive():
    print("called")
    return []

d = {"a": [1, 2]}
d.setdefault("a", expensive())          # prints "called" — even though "a" exists
d.setdefault("b", expensive())          # prints "called" — and uses the result
d.setdefault("a", expensive())          # prints "called" — wasted again

d["a"] exists from the start. setdefault returns the existing value. But Python evaluated expensive() to compute the argument before the call.

Why it happens

d.setdefault(k, default) is a regular method call. Python’s evaluation order:

1. Evaluate all arguments to the call (including default).
2. Pass them to the method.
3. Method decides whether to use default.

There’s no syntax-level lazy evaluation. The method can’t know to skip expensive() because by the time the method runs, the value is already computed.

This applies to dict.get(k, default) too — same trap.

How to avoid

defaultdict — factory called only on miss

from collections import defaultdict
d = defaultdict(list)              # `list` factory called only when key is missing
d["a"].append(1)
d["b"].append(2)

defaultdict(factory) calls factory() lazily, only when accessing a missing key. No wasted work.

Manual check

if k not in d:
    d[k] = expensive()
return d[k]

Or:

val = d.get(k)
if val is None:
    val = d[k] = expensive()

(Caveats: None could be a valid stored value. Use KEY_MISSING = object() sentinel if so.)

Memoization via functools.cache

If expensive() is pure and parameterized, lift it out:

import functools

@functools.cache
def expensive(key):
    return ...

d = {k: expensive(k) for k in keys}

The cache layer handles the lazy-compute, no setdefault needed.

When setdefault is fine

When the default is cheap — a literal [], 0, "", set():

groups = {}
for item in items:
    groups.setdefault(item.kind, []).append(item)

[] allocation is cheap; doing it on every call wastes a few microseconds. defaultdict(list) is still cleaner here, but both work.

Real-world bug

The classic version: lazy-loading a config or DB record into a cache.

def get_user(user_id):
    return _cache.setdefault(user_id, db.fetch_user(user_id))
    # db.fetch_user runs on EVERY call, even cache hits

Fix:

def get_user(user_id):
    if user_id not in _cache:
        _cache[user_id] = db.fetch_user(user_id)
    return _cache[user_id]

Or:

import functools
@functools.lru_cache(maxsize=1024)
def get_user(user_id):
    return db.fetch_user(user_id)

Interview angle

• Q: “What’s wrong with d.setdefault(k, expensive())?” — expensive() runs every call, even when k exists.
• Q: “When is setdefault still OK?” — cheap literal default ([], 0, "").
• Follow-up: “How does defaultdict differ?” — factory is called lazily on missing-key access; no wasted work for hits.
• Follow-up: “Why doesn’t Python optimize this?” — Python evaluates arguments before the call; the method can’t suppress already-evaluated arguments.

See 16_dict_get_falsy_default.md, 01_mutable_default_arguments.md, stdlib/01_collections.md.