Python memory model — allocation, refcounting, GC

CPython uses two garbage collection mechanisms in tandem:

1. Reference counting — primary, immediate
2. Generational cycle collector — handles reference cycles that refcount can’t

Reference counting

Every object has a count of how many references point to it. The count goes up on assignment, function call, container insert; goes down on del, function return, container removal.

import sys

x = []                      # refcount=1
y = x                       # refcount=2
sys.getrefcount(x)          # 3 (the third is the temporary in the call itself)
del y                       # refcount=1
del x                       # refcount=0 → object freed immediately

Pros: deterministic destruction — __del__ runs the moment refcount hits 0. RAII-style cleanup works.

Cons: cycles never get freed. Two objects pointing at each other have non-zero refcount even when nothing else references them:

a = []
b = []
a.append(b)
b.append(a)
del a, b   # refcounts still 1 each — leak without the cycle collector

Cycle collector (generational GC)

The gc module periodically walks heap objects, finds cycles, and collects them. It’s tuned for the generational hypothesis: most objects die young.

• Generation 0: newly-allocated container objects. Collected most often.
• Generation 1: objects that survived gen0. Collected occasionally.
• Generation 2: long-lived objects. Rarely collected.

When gen0 fills up, GC runs. Survivors promote to gen1. Same for gen1 → gen2.

import gc

gc.get_count()        # (gen0_alloc, gen1_alloc, gen2_alloc)
gc.collect()          # force a full collection — returns # objects collected
gc.get_threshold()    # (700, 10, 10) by default — gen0 trigger after 700 net allocs
gc.disable()          # turn off cycle GC (refcounting still runs)

GC overhead is small but real. Some workloads (large numerical pipelines) benefit from gc.disable() during hot loops.

Only container types participate in cycle GC

Cycle GC tracks only objects that can be part of a cycle: containers (list, dict, set, tuple-of-containers, custom classes with attributes). Atomic values (int, str, bool, float, simple bytes) don’t need it — they can’t reference other objects.

Use gc.is_tracked(obj) to check.

__del__ and cycles — the trap

If a class defines __del__ and is in a cycle, in Python <3.4 the cycle collector refused to collect because it couldn’t determine safe order to call finalizers. Python 3.4+ (PEP 442) fixed this — finalizers run, but in a one-shot manner per cycle.

Even so, avoid __del__: it’s hard to reason about, runs during interpreter shutdown unreliably, and exceptions in it are silently logged. Prefer context managers (__enter__/__exit__) and weakref.finalize.

Reference counting and threading

Refcount changes use the GIL — they’re atomic but not free. PEP 703 (free-threaded Python) replaces this with biased reference counting. Until that’s mainstream (3.13 experimental, optional), the GIL is what makes refcounting work without races.

Inspecting an object’s references

import gc

x = SomeObject()
gc.get_referrers(x)   # list of objects that reference x
gc.get_referents(x)   # list of objects x references

Useful for memory leak hunting — find what’s holding onto an object that should have been freed.

The allocator: arenas, pools, blocks

GC decides when memory is reclaimed. The allocator decides where objects live. CPython does not call C malloc for every small object — that’s slow and fragments the heap — so it ships its own small-object allocator, pymalloc (Objects/obmalloc.c).

Four layers

A request tries its type’s free list first (layer 3); failing that, pymalloc (layer 2) if it’s small; otherwise malloc (layer 1). PEP 445 lets you swap any layer’s allocator.

The hierarchy

pymalloc rounds each request up to a multiple of 8 bytes (ALIGNMENT), producing 64 size classes (exact alignment is platform/version-dependent — the model is unchanged):

request (bytes)   block size   size-class idx
    1–8               8              0
    9–16             16              1
   17–24             24              2
    …                 …             …
  505–512           512             63

All blocks in a pool share one size class, so allocation never searches for a fitting hole.

Allocating and freeing a small object

Each size class keeps a list of “used” (partially full) pools. Allocation pops a block from one; freeing pushes it back onto the pool’s free list. The trick: a freed block stores the pointer to the next free block in its own memory, so alloc/free are O(1) with no side metadata.

a = object()        # 16-byte instance, allocated by pymalloc
addr = id(a)
del a               # block returned to its pool's free list
b = object()        # same size class → reuses the freed block
id(b) == addr       # often True (CPython detail, not guaranteed)

A pool is used (partially full), full, or empty (free, not yet bound to a size class).

When memory returns to the OS — and when it doesn’t

An arena is released to the OS only when all 64 of its pools are free. A single live block keeps the whole 256 KB reserved. CPython sorts arenas by free-pool count and allocates from the most-used arena first, deliberately leaving others empty so they can be freed (CPython before 2.5 never returned arenas at all).

The consequence interviewers probe:

big = [object() for _ in range(10_000_000)]
del big              # objects freed, but RSS may barely move

Survivors elsewhere fragment the arenas, so the process stays large. Peak allocation sizes the process, not current usage. Objects > 512 B go through malloc, whose return-to-OS behavior is the C library’s call (glibc often retains freed memory too).

Object-specific free lists (layer 3)

For the hottest types CPython skips the allocator entirely:

• Small ints −5..256 are preallocated singletons; literals and identifiers are interned — see 30_string_interning.md.
• floats, lists, tuples, dicts, and frames keep free lists of recently-freed instances (which types and sizes shift between versions — 3.11/3.12 reworked several).
• __slots__ drops the per-instance __dict__ to shrink objects — see 28_slots.md; list over-allocation and dict compaction live in 33_list_internals.md and 32_dict_internals.md.

Inspecting the allocator

import sys, tracemalloc

sys.getsizeof([])           # bytes for one object (incl. GC header if tracked)
sys._debugmallocstats()     # dump pymalloc arena/pool/block stats to stderr

tracemalloc.start()
# ... run workload ...
for stat in tracemalloc.take_snapshot().statistics("lineno")[:5]:
    print(stat)             # top allocation sites by size

PYTHONMALLOC=debug turns on allocator debug hooks (catches overruns / use-after-free); PYTHONMALLOCSTATS=1 dumps stats at exit. Full leak-hunting workflow: performance/02_memory_profiling.md.

Interview angle

• “How does Python free memory?” Refcounting + cycle GC.
• “What happens to a cycle if cycle GC is disabled?” Leak.
• “Why does __del__ run immediately when an object goes out of scope?” Refcount → 0 — deterministic destruction.
• “Why doesn’t every refcount drop trigger Python’s GC?” Refcount drop is the primary GC; the cycle collector is an additional layer on top.
• “Does CPython malloc every object?” No — requests ≤ 512 bytes go through pymalloc’s arena/pool/block system; only larger ones reach the C allocator.
• “Difference between an arena, a pool, and a block?” Block = one object’s slot (8–512 B); pool = a 4 KB page of same-size blocks; arena = 256 KB of pools and the unit handed to/from the OS.
• “Why doesn’t memory drop after I del a big list?” An arena returns to the OS only when all its pools are free; surviving objects fragment arenas, so RSS is sticky — peak usage sizes the process.
• “What is pymalloc, and when is it bypassed?” CPython’s small-object allocator; bypassed for requests > 512 bytes, and replaceable via PEP 445.