The agent loop

The mechanism underneath every agent framework. Being able to write it from memory, and name where it breaks, is the practical test.

ReAct

Reason + Act. The model alternates between thinking and calling tools, feeding each observation back in.

Thought:      I need the customer's order history.
Action:       search_orders(email="a@b.com")
Observation:  [3 orders, most recent #1042 shipped 2 days ago]
Thought:      #1042 is the likely subject. Check its tracking.
Action:       get_tracking(order_id="1042")
Observation:  Delivered yesterday 14:20.
Thought:      I can answer now.
Answer:       Order #1042 was delivered yesterday at 14:20.

Modern implementations don’t parse Thought:/Action: from text — native tool calling returns structured calls. But the loop shape is the same, and “ReAct” still names it.

messages = [{"role": "user", "content": query}]

for step in range(MAX_STEPS):
    response = llm(messages, tools=TOOLS)
    messages.append(response)

    if not response.tool_calls:
        return response.content              # model decided it's done

    for call in response.tool_calls:
        result = execute(call)               # validate args before this
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": truncate(result),      # bound the observation
        })

raise AgentStepLimitExceeded(step)            # loud, not silent

Four things in that snippet that separate working code from a demo:

• MAX_STEPS — always. Without it a confused agent loops until your bill notices.
• truncate(result) — an unbounded tool result can blow the context window in one call.
• Validate arguments before executing. The model produces plausible-looking arguments that don’t always satisfy your schema.
• Fail loudly on step exhaustion. Silently returning a partial answer hides the failure.

Termination

Deciding when to stop is where naive loops break.

An explicit finish(answer) tool is underrated: it makes “I’m done” an action the model takes rather than an absence you infer, and it lets you enforce a schema on the final answer.

Loop detection

The most common runtime failure is an agent repeating the same call because the result isn’t what it wanted.

seen = collections.Counter()

key = (call.name, json.dumps(call.args, sort_keys=True))
seen[key] += 1
if seen[key] > 2:
    # Don't just abort - tell the model what's happening
    messages.append({
        "role": "tool", "tool_call_id": call.id,
        "content": f"You have called {call.name} with these arguments "
                   f"{seen[key]} times and received the same result. "
                   f"Try a different approach or explain what's blocking you.",
    })
    continue

Feeding the observation back is better than aborting — the model frequently recovers when told it’s stuck. Aborting turns a recoverable situation into a failure.

Parallel tool calls

Models can return several tool calls in one response. Executing them concurrently is usually free latency:

async with asyncio.TaskGroup() as tg:
    tasks = {c.id: tg.create_task(execute_async(c)) for c in response.tool_calls}

Two cautions: tools with side effects may not be safe to run concurrently, and every result must be appended — a missing tool_call_id response breaks the message sequence with most providers. See ../../backend/04_async_concurrency/12_taskgroup_structured_concurrency.md.

Context growth

Every observation is appended, so the prompt grows monotonically. By step 10 you may be re-sending a large transcript on every call.

Mitigations, roughly in order of preference:

• Return less. Summarise or filter tool output at the tool, not afterwards.
• Truncate old observations, keeping the most recent verbatim.
• Summarise middle history while keeping the first message (the task) and recent turns.
• Externalise — write large results to a store, put a reference in context.
• Sub-agents — delegate a subtask to a fresh context and return only its conclusion.

Note that rewriting history invalidates prefix caching from the point of change, so a summarisation that saves tokens can cost more in re-prefill. Measure both. See ../06_transformers_llm/05_kv_cache.md and ../12_context_engineering/.

Planning variants

Plan-and-execute is genuinely useful when you want the plan reviewable before anything executes — it turns an opaque process into something a human can approve.

Observability

An agent without tracing is undebuggable. Log per step: the model call, its inputs and outputs, tool name and arguments, result, latency, tokens, cost.

with tracer.start_span("agent.step", attributes={"step": step}):
    ...

OpenTelemetry has GenAI semantic conventions for exactly this, so agent traces sit alongside your normal service traces rather than in a separate tool. LangSmith, Langfuse and similar build on the same idea. See ../../backend/15_observability/07_opentelemetry.md.

Interview angle

• “Describe the agent loop.” — call the model with the tool schemas; if it returns tool calls, execute them and append the results as tool messages; repeat until it returns a plain answer or a limit trips. ReAct is the name for the reason-then-act alternation.
• “How do you stop an agent looping forever?” — a step limit and a cost/time budget as hard stops, plus loop detection on repeated identical calls. Feed the detection back to the model as an observation rather than aborting; it often recovers.
• “What blows up the context in an agent?” — unbounded tool results. Truncate or summarise at the tool boundary, externalise large payloads, and note that rewriting history to save tokens invalidates the prefix cache.
• “The model returns three tool calls at once. What do you do?” — execute concurrently where the tools are side-effect-free, and append a result for every tool_call_id — a missing one breaks the message sequence.
• “ReAct vs plan-and-execute?” — ReAct decides one step at a time and adapts; plan-and-execute commits to a plan up front, which is cheaper and auditable but brittle when reality diverges. Plan-first is valuable when a human should approve before anything runs.
• “How do you debug an agent that gave a wrong answer?” — per-step traces of model input/output, tool calls, arguments, results, latency and tokens. Without that you’re guessing. OpenTelemetry GenAI conventions put it in the same tracing system as everything else.