backend / async concurrency / 06_event_loop_internals.md

Event Loop Internals

9 interview angles 7 min read source

Event Loop Internals

What’s actually running when you await. The event loop is a single thread that round-robins between ready coroutines, sleeps on I/O via the OS, and wakes them when ready.

The simplified loop

while running:
    1. Run all ready callbacks (scheduled, fired, completed I/O)
    2. Process ready coroutines until they `await` something
    3. Compute the next timeout (next scheduled timer or 0 if there's I/O ready)
    4. Poll the OS for ready file descriptors (selectors / epoll / kqueue)
    5. Mark callbacks ready for any returned FDs

Each iteration is one “tick.” Coroutines run cooperatively — they yield at await points, and only then can the loop switch to another coroutine.

await is a yield point

async def f():
    print("a")
    await asyncio.sleep(0)      # yield to the loop
    print("b")
    x = await fetch()           # yield until fetch completes
    print("c")

Between awaits, the function runs synchronously. Other coroutines do not interleave inside that block. If you have a tight CPU loop with no await, you block the loop just as badly as time.sleep.

Where time goes

  • asyncio.sleep(t) — schedules a callback for t seconds in the future; yields to the loop. The loop sleeps in epoll_wait until either I/O is ready or the timer fires.
  • await read_socket() — registers the FD with the selector for “readable”; yields. When the OS marks the FD ready, the loop resumes the coroutine.
  • asyncio.wait_for(coro, t) — runs the coro; schedules a cancel callback at t. On timeout, cancels and raises TimeoutError.

Where coroutines live

Each coroutine is a state machine. await saves its state and yields a “future” the loop tracks. When the future resolves, the loop sends the result back into the coroutine, which resumes from the saved state.

You don’t see the state machine — Python’s async/await syntax compiles to it. But it’s why coroutines have no per-task thread, no per-task stack overhead beyond the function frame: they’re just Python objects with a __next__ (technically .send).

get_event_loop vs get_running_loop vs run

Function Use
asyncio.run(coro) top-level entry; creates a new loop, runs, closes. Use this.
asyncio.get_running_loop() from inside a running coroutine; gets the current loop.
asyncio.get_event_loop() deprecated for new code (3.10+). Tries running loop first, falls back to creating one — that fallback is the bug source.
asyncio.new_event_loop() only in test harnesses or threaded loop setups.
# Top level
async def main():
    ...

asyncio.run(main())     # standard

Don’t reach for get_event_loop() unless you’re maintaining old code. get_running_loop() from inside async, asyncio.run at the top.

Why blocking the loop is catastrophic

Single thread + cooperative scheduling = if one coroutine doesn’t yield, nothing else runs. Concretely:

@app.get("/slow")
async def slow():
    time.sleep(2)             # blocks loop for 2s
    return {"ok": True}

While this runs, every other concurrent request waits. With 100 concurrent users hitting /slow, the 100th waits 200 seconds. There’s no thread to switch to.

Diagnostic: enable asyncio’s debug mode in development.

asyncio.run(main(), debug=True)

It warns when callbacks take > 100ms — the typical “you blocked the loop” signal.

Tasks vs coroutines

  • Coroutine — what async def returns when called. Inert; doesn’t run until awaited.
  • Task — a coroutine wrapped and scheduled by the loop. Runs in the background.
async def bg():
    await asyncio.sleep(5)
    print("done")

# Coroutine — never scheduled, never runs
co = bg()

# Task — scheduled immediately
task = asyncio.create_task(bg())
await task        # waits for completion

Forgetting to schedule a coroutine is a silent bug — Python’s “coroutine was never awaited” warning catches some cases.

Future objects

A Future is the low-level synchronization primitive — “a value that will be available later.” Tasks are Futures plus a coroutine driver.

You rarely create Futures directly. Most code uses tasks. Use Future when bridging non-asyncio APIs (callback-based libraries):

def fetch_callback_style(cb):
    # legacy library; calls cb(result) when done
    ...

async def fetch_async():
    loop = asyncio.get_running_loop()
    fut = loop.create_future()
    fetch_callback_style(lambda result: fut.set_result(result))
    return await fut

call_soon, call_later, call_at

Schedule callbacks (not coroutines) on the loop:

loop = asyncio.get_running_loop()
loop.call_later(5.0, callback, arg)          # in 5s
loop.call_at(loop.time() + 5.0, callback)    # absolute time
loop.call_soon(callback, arg)                # next iteration

Used internally by the asyncio machinery; occasionally useful for “schedule cleanup later” patterns.

Multiple loops / threading

By default, one event loop per thread. The loop is not thread-safe — calling task.cancel() from another thread is undefined.

For cross-thread interaction:

  • loop.call_soon_threadsafe(callback, *args) — schedule callback from another thread.
  • asyncio.run_coroutine_threadsafe(coro, loop) — schedule a coroutine; returns a concurrent.futures.Future you can wait on from the calling thread.

Selector backends

asyncio uses the OS’s best I/O multiplexer:

OS Backend
Linux epoll
BSD / macOS kqueue
Windows IOCP (via ProactorEventLoop)

On Windows, the default is ProactorEventLoop (3.8+) — it uses IOCP and supports subprocess I/O. The older SelectorEventLoop is available but deprecated for general use.

uvloop

A drop-in replacement loop written in C (libuv). 2-4× faster than the default for I/O-heavy workloads. Linux/macOS only.

import uvloop
uvloop.install()        # or uvloop.run(main()) on 3.11+

asyncio.run(main())     # now uses uvloop

Used by uvicorn, FastAPI in production. Same API.

Python vs JavaScript event loops

A frequent interview comparison, especially in full-stack rounds.

Python (asyncio) JavaScript (Node.js)
Loop lifetime explicit — you call asyncio.run() implicit — always running, part of the runtime
Scheduling coroutines and Tasks callback queue + microtask queue
Yield point await await / .then()
Concurrency model cooperative, single-threaded cooperative, single-threaded
CPU-bound work blocks the loop; offload to thread/process blocks the loop; offload to worker threads

The one that gets asked: why doesn’t Python start a loop automatically? Because Python supports several concurrency models — threads, processes, asyncio — and the runtime doesn’t presume you want asyncio. Node has exactly one model, so its loop is always live. That’s why Python needs asyncio.run(main()) and JavaScript doesn’t.

Both are cooperative and single-threaded, so the blocking-call failure mode is identical in both languages.

One loop per thread

You can create multiple loops with asyncio.new_event_loop(), but the rule is one running loop per thread. Two loops in the same thread is unsupported. Different threads may each have their own loop, which is how you’d embed asyncio inside a threaded application.

The loop itself is not thread-safe. To hand work to a loop from another thread, use asyncio.run_coroutine_threadsafe(coro, loop) — it returns a concurrent.futures.Future, not an asyncio one. Going the other way, to run blocking code without stalling the loop, use await asyncio.to_thread(fn) or loop.run_in_executor(...) — see 37_run_in_executor_to_thread.md.

Production tips

  • Always set timeouts on external calls. The loop is happy to sleep forever on a slow peer.
  • Profile with asyncio.run(..., debug=True) during dev. Warnings about slow callbacks identify blockers.
  • Avoid time.sleep, requests, sync DB drivers in coroutines. Use asyncio.sleep, httpx, asyncpg/SQLAlchemy async.
  • Use TaskGroup (3.11+) for fan-out — no leaked tasks.
  • Run uvloop in prod for I/O-heavy services.

Interview angle

  • “What is the event loop doing while a coroutine is awaiting I/O?” — sleeping in epoll_wait (or kqueue/IOCP) until either a watched FD becomes ready or a timer fires. When something’s ready, the loop wakes, runs scheduled callbacks, resumes the coroutine.
  • “Why is calling time.sleep in async code so bad?” — single-threaded loop. The sleep doesn’t yield; every other coroutine is blocked until the sleep returns. Use asyncio.sleep.
  • asyncio.get_event_loop vs get_running_loop vs asyncio.run?”run is the top-level entry (creates+runs+closes); get_running_loop is the in-coroutine accessor (raises if no loop); get_event_loop is deprecated for new code (silently creates a loop in some cases — the bug source).
  • “Coroutine vs Task?” — coroutine: inert object returned by calling async def. Task: scheduled coroutine wrapped by the loop. asyncio.create_task upgrades coroutine → task; gather and TaskGroup do this for you.
  • “What’s uvloop and when do you use it?” — drop-in C-based event loop (libuv). 2-4× faster than the stdlib for I/O-heavy workloads. Production default for FastAPI / uvicorn on Linux/macOS.
  • “How does cancellation work?”task.cancel() schedules CancelledError to be thrown into the coroutine at its next await point. The coroutine can catch it for cleanup, but should re-raise (or asyncio re-raises automatically). Depth on this in 21_cancellation_timeouts_errors.md.
  • “Why doesn’t Python run an event loop automatically like Node?” — Python supports threads, processes and asyncio; the runtime doesn’t assume which you want. Node has one concurrency model, so its loop is always running.
  • “Can you have more than one event loop?” — one running loop per thread. Multiple threads can each have one. The loop is not thread-safe; cross-thread submission goes through asyncio.run_coroutine_threadsafe.
  • “What is cooperative multitasking here?” — tasks yield voluntarily at await; the loop never preempts them. That’s exactly why one un-awaited CPU-bound stretch stalls every other coroutine.