Agent Patterns — Plan-and-Execute, Reflection, Cost, Evaluation

The patterns interview questions actually probe, beyond ReAct and “LangGraph syntax”. The senior view: agents are systems with cost, failure modes, and evaluation needs — not just a clever loop.

Plan-and-Execute vs ReAct

ReAct

Tightly interleaved Thought-Action-Observation loop:

Thought: I need to find the user's order count.
Action: query_db({"sql": "SELECT COUNT(*) FROM orders WHERE user_id=42"})
Observation: 17

Thought: Now I'll check their tier.
Action: query_db({"sql": "SELECT tier FROM users WHERE id=42"})
Observation: gold

Thought: I have the data.
Answer: User 42 has 17 orders and is in the gold tier.

The LLM decides the next step at each iteration. Flexible — adapts to results. Drawback: the LLM has to think about both what to do next and what the high-level plan is on every iteration. For long chains, this often loops, backtracks, or loses track.

Plan-and-Execute

Two-phase: first plan the whole task, then execute steps.

Phase 1 (planner): Plan to answer "Is user 42 eligible for VIP?"
  - Step 1: Get user 42's tier
  - Step 2: Get user 42's lifetime spend
  - Step 3: Check eligibility rules (tier=gold AND spend>10000)

Phase 2 (executor): execute Step 1 → tier=gold
                    execute Step 2 → spend=$15,000
                    execute Step 3 → eligible: yes

Wins:

• Plan is generated once; cheaper than re-planning every step.
• Plan is auditable; users can see and approve.
• Less prone to loops (executor doesn’t deviate from the plan).
• Parallelizable: independent steps can run concurrently.

Drawbacks:

• Plans can be wrong; replanning needed when execution diverges.
• Less adaptive — if step 2’s result invalidates step 3, you need a replan loop.

When each

Production pattern: Plan-and-Execute as the default, ReAct sub-loop for the “execute” of any step that’s itself dynamic.

LangChain has create_plan_and_execute_agent; the LangGraph approach is often a custom planner node + executor sub-graph.

Reflection / Self-Critique

After producing an answer, ask the LLM to critique it:

1. Generate initial response.
2. Critique: "Is this answer correct? Are there errors? What's missing?"
3. If critique finds issues: regenerate with the critique as feedback.
4. Repeat up to N times.

async def reflect(question, max_iterations=3):
    response = await llm.generate(f"Answer: {question}")
    for i in range(max_iterations):
        critique = await llm.generate(
            f"Critique this answer for accuracy, completeness, clarity:\n\nQ: {question}\nA: {response}\n\nIf the answer is good, say 'ACCEPTED'."
        )
        if "ACCEPTED" in critique:
            return response
        response = await llm.generate(
            f"Revise this answer based on the critique.\n\nQ: {question}\nA: {response}\nCritique: {critique}"
        )
    return response

Reflection works best for:

• Code generation (catch syntax errors, missing imports, logic bugs).
• Long-form writing (factual checking, completeness).
• Tasks where the model can recognize errors better than it avoids them.

Doesn’t help:

• Tasks the model can’t ground externally (the model’s wrong “facts” are also wrong in the critique).
• Simple tasks where the first answer is fine (wasted tokens).

Reflection variants

Self-Refine — generic reflection loop on the model’s own output.

Self-Discover (Google, 2024) — model first picks reasoning patterns, then applies them.

Self-Consistency — generate N answers, vote on majority. Stronger than single reflection for arithmetic / logic; weaker for open-ended.

Reflexion — adds a “memory” of past attempts; the model learns from prior failures within a session.

Cost control

Agents can rack up huge token bills. Patterns to bound it:

Per-iteration token budget

class TokenBudgetExceeded(Exception):
    pass

class TrackingClient:
    def __init__(self, max_tokens):
        self.used = 0
        self.max = max_tokens

    async def generate(self, prompt, ...):
        response = await llm.generate(prompt, ...)
        self.used += response.usage.total_tokens
        if self.used > self.max:
            raise TokenBudgetExceeded()
        return response

Set a hard cap on tokens per request. Beyond it, abort or fall back to a cheaper path.

Max iteration cap

MAX_ITERATIONS = 10

for i in range(MAX_ITERATIONS):
    response = await llm.generate(...)
    if response.is_final():
        break
else:
    raise AgentLoopExceeded()

Always cap. Without it, a confused model can loop forever on tool calls.

Smaller model for sub-tasks

async def agent(question):
    # Big model for planning
    plan = await openai.chat.completions.create(model=MODEL, ...)

    # Cheap model for execution sub-steps
    for step in plan.steps:
        result = await openai.chat.completions.create(model=SMALL_MODEL, ...)

Planner makes the high-stakes decisions; executor handles cheap mechanical steps with a smaller model. Often 5-10× cost reduction with minimal quality loss.

Prompt caching

Anthropic / OpenAI / Bedrock support prompt caching. Static prefixes (system prompt, tool definitions, few-shot examples) cached on the server; subsequent calls only pay for the new tokens.

response = anthropic.messages.create(
    model="claude-sonnet-4-5",
    system=[
        {"type": "text", "text": LONG_STATIC_SYSTEM_PROMPT, "cache_control": {"type": "ephemeral"}},
    ],
    messages=[...],
)

The system prompt is cached (5 min ephemeral). Subsequent calls within 5 min skip re-processing it. 10× cheaper, much faster.

For agent loops with large fixed prompts (tool definitions, examples, system rules), prompt caching is a huge win.

Tool-result truncation

Tool outputs accumulate in the conversation history; long histories = expensive.

# After tool call, summarize result
result = call_tool(...)
if len(result) > 1000:
    summary = await summarize(result)
    history.append({"role": "tool", "content": summary})
else:
    history.append({"role": "tool", "content": result})

Summarize aggressive long tool outputs (DB rows, web pages) before they go into the context. Trade some fidelity for huge cost savings.

Cost monitoring

import structlog
log = structlog.get_logger()

async def agent_run(question):
    start = time.time()
    tokens_used = 0
    iterations = 0
    try:
        # ... agent loop ...
        return result
    finally:
        log.info(
            "agent_complete",
            question=question,
            tokens=tokens_used,
            iterations=iterations,
            duration_ms=int((time.time() - start) * 1000),
            cost_usd=tokens_used * COST_PER_TOKEN,
        )

Track per-request cost. Identify outliers; cap and alert.

Evaluation of agents

Single-shot model evaluation (BLEU, ROUGE, perplexity) doesn’t capture agent quality. Agent evaluation focuses on:

Task success rate

Did the agent achieve the goal?

• For closed-ended tasks (math problems, code generation with tests): check programmatically.
• For open-ended tasks: LLM-as-judge or human evaluation.

Plan quality

Independent of final answer: was the plan good?

• Were the right tools chosen?
• Did the agent skip necessary steps?
• Were there redundant calls?

LLM-as-judge scoring on a rubric:

JUDGE_PROMPT = """
Rate this agent's execution on:
- Plan correctness (1-5): did it choose the right approach?
- Efficiency (1-5): were there unnecessary steps?
- Result correctness (1-5): is the final answer right?

Trace: {trace}
Final answer: {answer}
Ground truth: {ground_truth}
"""

Step-level evaluation

For Plan-and-Execute: evaluate each step independently. A correct final answer can mask wrong individual steps that happened to land on the right answer.

Frameworks

Continuous evaluation

Set up evals to run on every code change, not just at launch:

# .github/workflows/agent-eval.yml
- name: Run agent eval
  run: |
    pytest tests/test_agent_eval.py
    # Each test: run agent on a fixed input; check tool usage + final answer

Regressions in agent quality become catchable in CI rather than discovered in production.

When NOT to use an agent

Agents add cost, complexity, and failure modes. Don’t reach for one when:

• Single LLM call suffices. Question → answer with optional retrieval. No tools needed.
• The flow is fully deterministic. State machines (Step Functions, Temporal) beat agents for known workflows.
• Latency budget is tight. Multi-step agents take seconds; users expecting sub-second responses notice.
• Cost budget is tight. Agents are 3-30× more expensive than single calls.
• High-reliability requirement. Agents fail in creative ways; deterministic code is more predictable.

The agent pattern shines when:

• The task genuinely requires multiple tool calls with branching.
• The flow can’t be pre-determined.
• The user trades latency for capability.
• Cost is acceptable per task.

Interview angle

• “ReAct vs Plan-and-Execute — when each?” — ReAct: interleaved reasoning and action, adaptive, good for short / exploratory tasks. Plan-and-Execute: separate planning phase, executor follows the plan; better for longer / structured tasks, auditable, parallelizable. Common production pattern: Plan-and-Execute outer loop with ReAct inner for dynamic sub-steps.
• “What’s reflection in agents?” — generate → critique → regenerate loop. Works best when the model can recognize errors better than it avoids them (code with bugs, factual errors). Doesn’t help on tasks the model can’t ground externally.
• “How do you control agent cost?” — token budget per request, max iteration cap, smaller model for sub-tasks (planner big, executor small), prompt caching for static prefixes, tool-output summarization to bound context growth, cost monitoring with per-request logging.
• “How do you evaluate an agent?” — task success rate (programmatic for closed tasks, LLM-as-judge for open), plan quality (rubric scoring), step-level correctness (not just final answer). Frameworks: LangSmith, Langfuse, Promptfoo. Continuous eval in CI catches regressions.
• “What’s prompt caching and when does it help agents?” — server-side caching of static prompt prefixes (system prompt, tools, examples). Subsequent calls within ~5 min skip re-processing the cached portion. Big cost + latency win for agent loops with large fixed prompts.
• “When not to use an agent?” — single LLM call suffices, deterministic flow (use Step Functions / Temporal instead), tight latency budget, tight cost budget, high reliability needs. Agents trade predictability for capability.
• “How do you stop an agent loop from running forever?” — max iteration cap (typical: 10-20), token budget cap, detection of repeated tool calls without progress, timeout on the whole request. Without these, a confused model can loop indefinitely.