run_in_executor and asyncio.to_thread

Async Python is single-threaded. The moment you call time.sleep, requests.get, or any sync I/O / CPU-bound function inside a coroutine, you block the event loop. The fix: offload to a thread or process pool.

The event loop blocking problem

@app.get("/users/{id}")
async def get_user(id: int):
    user = db.query(User).get(id)        # blocking sync DB call — BLOCKS EVENT LOOP
    return user

While the sync call runs (even for 50ms), every other request handler in the process is stuck. Throughput collapses; tail latency explodes. This is the #1 production gotcha for new async Python services.

The fix: a thread

asyncio.to_thread (3.9+) runs a sync function in the default thread pool:

@app.get("/users/{id}")
async def get_user(id: int):
    user = await asyncio.to_thread(db.query(User).get, id)
    return user

The event loop hands the call to a worker thread, awaits, returns. Other coroutines run while it waits.

Lower-level: run_in_executor

import asyncio

loop = asyncio.get_running_loop()
result = await loop.run_in_executor(None, sync_function, arg1, arg2)

None means “use the default executor” (a ThreadPoolExecutor). For CPU-bound work, pass a ProcessPoolExecutor:

from concurrent.futures import ProcessPoolExecutor

with ProcessPoolExecutor() as pool:
    result = await loop.run_in_executor(pool, cpu_heavy_function, big_input)

asyncio.to_thread is a thin wrapper over run_in_executor(None, ...). Use it for thread-pool offload; use run_in_executor when you need a specific executor (process pool, custom pool size).

Thread pool vs process pool

Threads release the GIL for I/O (sockets, files, sleeps); CPU-bound code in threads is GIL-bound and won’t parallelize.

Sizing the default thread pool

In Python 3.13 and earlier, default is min(32, os.cpu_count() + 4). For an async service hitting a sync DB driver heavily, this is often too small:

import asyncio
from concurrent.futures import ThreadPoolExecutor

# Custom pool for sync DB calls
db_pool = ThreadPoolExecutor(max_workers=50, thread_name_prefix="db")

async def get_user(id):
    loop = asyncio.get_running_loop()
    return await loop.run_in_executor(db_pool, db.query, id)

Rule of thumb: pool size ≥ peak concurrent sync calls you expect. Otherwise queued requests stall.

FastAPI’s helper: run_in_threadpool

FastAPI auto-detects sync def route handlers and runs them in a threadpool. For helper functions you call yourself:

from fastapi.concurrency import run_in_threadpool

@app.get("/report")
async def report():
    data = await run_in_threadpool(generate_report_sync)
    return data

Equivalent to asyncio.to_thread but works on older Python and is the FastAPI idiom.

Sync DB driver in an async route

The most common case. Two options:

Option A (lazy fix): keep the sync driver, wrap calls in to_thread.

@app.get("/users")
async def list_users():
    return await asyncio.to_thread(_sync_list_users)

Option B (proper fix): use an async driver — asyncpg, aiomysql, SQLAlchemy 2.0 async.

@app.get("/users")
async def list_users(session: AsyncSession = Depends(get_session)):
    return (await session.execute(select(User))).scalars().all()

B is better long-term. A is the bridge while you migrate.

CPU-bound: use a process

GIL means thread pool won’t help CPU-bound code. Go to processes:

import asyncio
from concurrent.futures import ProcessPoolExecutor

executor = ProcessPoolExecutor(max_workers=4)

async def compute(data):
    loop = asyncio.get_running_loop()
    return await loop.run_in_executor(executor, heavy_cpu_work, data)

Caveats:

• Args + return must pickle.
• Memory cost — each process is a separate Python interpreter.
• Slow startup; reuse the pool, don’t spawn per call.

For truly heavy CPU, consider: dedicated worker service (Celery), native code (NumPy/Cython), or moving to a language that doesn’t have a GIL for that one piece.

contextvars and threads

contextvars (used by OpenTelemetry, structlog request context) don’t propagate to threads automatically. Pass context explicitly:

import contextvars

ctx = contextvars.copy_context()
result = await loop.run_in_executor(None, ctx.run, sync_fn, arg)

asyncio.to_thread does this for you (since 3.9). run_in_executor does NOT — you must copy context yourself if you care about trace propagation, log correlation IDs, etc.

Gotchas

• Calling sync code from a coroutine directly. No await, no threadpool — just freezes the loop. Easy mistake.
• Forgetting await on asyncio.to_thread. Returns a coroutine; without await nothing runs.
• Thread pool exhaustion silently queues. If your pool is 32 and you have 200 concurrent sync DB calls, requests pile up. Set pool size deliberately.
• run_in_executor doesn’t copy contextvars on older Python. OpenTelemetry traces dead-end into threads. Use to_thread or copy explicitly.
• Mixing async and sync DB drivers in one app. Two connection pools, two sources of contention, double the surface area. Pick one.

Quick reference

# Modern idiom
result = await asyncio.to_thread(sync_function, *args, **kwargs)

# Custom executor
result = await loop.run_in_executor(executor, sync_function, *args)

# FastAPI helper (older Python or FastAPI idiom)
result = await run_in_threadpool(sync_function, *args, **kwargs)

# CPU-bound
result = await loop.run_in_executor(process_pool, cpu_function, data)

Interview angle

• “You call time.sleep(1) in an async route. What happens?” — blocks the entire event loop. Every other coroutine in the process is stuck for 1 second. Use await asyncio.sleep(1) instead.
• “How do you call a sync DB driver from an async FastAPI route?” — wrap it in asyncio.to_thread (or FastAPI’s run_in_threadpool). It runs in a thread pool; the event loop is free to handle other requests. Proper fix: use an async DB driver.
• “asyncio.to_thread vs run_in_executor?” — to_thread is a thin wrapper over run_in_executor(None, fn, ...). Use to_thread for the default thread pool; run_in_executor when you need a custom executor (process pool, sized thread pool).
• “How do you handle CPU-bound work in async Python?” — run_in_executor with a ProcessPoolExecutor. GIL means thread pool won’t help CPU-bound code. Args must pickle. Reuse the pool — don’t spawn per call.
• “Why does OpenTelemetry trace context get lost in run_in_executor?” — contextvars don’t propagate to threads automatically by raw run_in_executor. asyncio.to_thread propagates them; manual run_in_executor requires contextvars.copy_context().run(...).
• “Threadpool exhaustion — what does it look like?” — sync calls queue waiting for a worker thread; latency rises but no error. Symptom: high p99 latency with low CPU. Fix: size the pool to your peak concurrent sync calls.