Function Calling, Tool Use, and Structured Output

The two patterns that turn LLMs from “produces text” into “does things.” Function calling lets the LLM call your code; structured output gives you typed responses you can use programmatically.

Function calling — what and why

LLM is asked a question. Instead of generating freeform text, it decides “I should call function X with these arguments.” Your code runs X, returns the result; the LLM uses the result to compose its actual answer.

User: "What's the weather in Tokyo?"
Model: { "tool": "get_weather", "args": {"city": "Tokyo"} }
Your code: get_weather("Tokyo") → "18°C, partly cloudy"
Model: "It's 18°C and partly cloudy in Tokyo right now."

The LLM picks the right tool, formats the args correctly, and integrates results into responses. You provide the tools; the model orchestrates.

OpenAI function calling

from openai import OpenAI
client = OpenAI()

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name"},
                "units": {"type": "string", "enum": ["c", "f"], "default": "c"},
            },
            "required": ["city"],
        },
    },
}]

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
)

msg = response.choices[0].message

if msg.tool_calls:
    for tc in msg.tool_calls:
        args = json.loads(tc.function.arguments)
        result = get_weather(**args)
        # Send the result back
        messages.append(msg)
        messages.append({
            "role": "tool",
            "tool_call_id": tc.id,
            "content": json.dumps(result),
        })

    # Second round — model uses the tool result
    response = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        tools=tools,
    )

The model can call multiple tools in parallel. Loop until it stops requesting tools.

Anthropic tool use

Equivalent pattern; different shape:

import anthropic
client = anthropic.Anthropic()

tools = [{
    "name": "get_weather",
    "description": "Get current weather for a city.",
    "input_schema": {
        "type": "object",
        "properties": {"city": {"type": "string"}, "units": {"type": "string", "enum": ["c", "f"]}},
        "required": ["city"],
    },
}]

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    tools=tools,
    messages=[{"role": "user", "content": "Weather in Tokyo?"}],
)

# response.content is a list of blocks: text and/or tool_use
for block in response.content:
    if block.type == "tool_use":
        result = get_weather(**block.input)
        # Send result back via a "tool_result" message
        followup = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            tools=tools,
            messages=[
                {"role": "user", "content": "Weather in Tokyo?"},
                {"role": "assistant", "content": response.content},
                {"role": "user", "content": [{
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "content": json.dumps(result),
                }]},
            ],
        )

Structured output

Sometimes you don’t want the model to call a tool — you want it to return structured data. “Extract these fields from this text” or “classify this into one of these categories.”

OpenAI: response_format=json_schema

from pydantic import BaseModel

class ExtractedOrder(BaseModel):
    customer_email: str
    items: list[str]
    total: float
    urgent: bool

response = client.chat.completions.parse(
    model=MODEL,   # pin the exact version in config
    messages=[
        {"role": "system", "content": "Extract order details from the email."},
        {"role": "user", "content": email_text},
    ],
    response_format=ExtractedOrder,
)

order = response.choices[0].message.parsed   # ExtractedOrder instance

The model is forced to emit JSON conforming to the schema. Strict mode ensures the output validates — no need to retry on malformed JSON.

Anthropic: tool use as forced JSON

Anthropic doesn’t have native “json mode”; the canonical pattern is to define a tool, force its use, and parse its arguments:

tools = [{
    "name": "submit_order",
    "description": "Submit the extracted order.",
    "input_schema": ExtractedOrder.model_json_schema(),
}]

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    tools=tools,
    tool_choice={"type": "tool", "name": "submit_order"},  # forces use
    messages=[{"role": "user", "content": f"Extract order from:\n\n{email_text}"}],
)

for block in response.content:
    if block.type == "tool_use":
        order = ExtractedOrder.model_validate(block.input)

When function calling vs structured output

They’re not mutually exclusive — a tool’s arguments are themselves structured output. The distinction is whether your code does something with the call.

Tool design

Good tools:

• Narrow, well-named. get_weather(city), not lookup(query).
• Idempotent where possible. The model may call the same tool twice; results should be safe to redo.
• Strong validation in your handler. Don’t trust the model to send well-formed args; validate.
• Clear descriptions. The model picks tools by description; vague descriptions → wrong tools used.
• Return JSON, not prose. The model needs structured data to reason about.

Bad tools:

• “Do anything” tools (execute_python_code, query_database) — too broad; hard for the model to use correctly; security nightmare.
• Stateful tools that depend on earlier calls.
• Tools with overlapping responsibilities (get_user vs lookup_user).

Tool errors

Tools can fail (API timeout, bad input, rate limit). Surface the error:

try:
    result = get_weather(**args)
    content = json.dumps({"ok": True, "data": result})
except WeatherAPIError as e:
    content = json.dumps({"ok": False, "error": str(e)})

messages.append({"role": "tool", "tool_call_id": tc.id, "content": content})

The model sees the error and can decide to retry, try a different tool, or apologize to the user. Don’t raise back to the model loop — let it handle the failure semantically.

Loop control

The agent loop is model → tool → model → tool → ... until the model responds without requesting a tool. Add safeguards:

MAX_ITERATIONS = 10

for i in range(MAX_ITERATIONS):
    response = client.chat.completions.create(...)
    if not response.choices[0].message.tool_calls:
        break  # done — model returned final answer
    for tc in response.choices[0].message.tool_calls:
        # execute tool
        ...
else:
    raise RuntimeError("Agent exceeded max iterations — likely looping")

Without this, a confused model can loop indefinitely (calling tools repeatedly without progress). 10-20 iterations is a reasonable bound.

Parallel tool calls

Modern models can request multiple tools in one response:

{
  "tool_calls": [
    {"name": "get_weather", "args": {"city": "Tokyo"}},
    {"name": "get_weather", "args": {"city": "Berlin"}},
    {"name": "get_time", "args": {"city": "Tokyo"}}
  ]
}

Execute them in parallel — much faster than sequential. asyncio.gather:

results = await asyncio.gather(*[
    execute_tool(tc) for tc in response.choices[0].message.tool_calls
])

Cost / latency considerations

• Each tool round trip = one LLM call. 5 tools sequentially = 5 LLM calls.
• Parallel tools collapse to one round-trip (the LLM call + one batched tool execution + one more LLM call).
• Tool results live in the conversation history — summarize / truncate to manage cost.
• Use cheaper models for tool selection if quality permits (smaller model decides which tool; bigger model composes the final answer).

MCP — Model Context Protocol

Anthropic’s standard (2024) for exposing tools to AI models in a portable way. An MCP server exposes tools / resources; the model connects to the server, discovers what’s available, calls tools. Standardizes how Claude (and increasingly other models) connect to external systems without bespoke integration code.

If you’re building tools to be consumed by multiple AI products, MCP is the emerging standard.

Common pitfalls

• Trusting tool args without validation. The model may emit wrong types, missing fields, hallucinated values. Validate with Pydantic; reject obviously bad input.
• No iteration cap. Loops can run forever; cap iterations.
• Tool descriptions copy-pasted from docs. Models read descriptions; vague descriptions → wrong tools used.
• Forgetting to send tool results back. Model called a tool, you executed it, but didn’t include the result in the next message → model has no idea what happened.
• Confused tool_call_id matching. Each tool result must reference the correct tool_call_id. Mismatches → “I see you called X but the result is for Y” weirdness.
• Returning huge tool results. Tokens add up. Truncate / summarize tool outputs before sending back.

Interview angle

• “What’s function calling / tool use?” — pattern where the LLM, given a list of tool schemas, decides to “call” one with arguments. Your code executes the call and returns the result; the model uses it to compose the next response. Loop until no more tools requested.
• “What’s structured output?” — make the LLM return JSON conforming to a schema, validated. OpenAI: response_format=YourPydanticModel. Anthropic: force a tool with the schema. Eliminates “parse-and-retry” on freeform text.
• “How do you handle tool errors?” — catch the exception in your handler; return a JSON error result back to the model. The model can then retry, try a different tool, or apologize. Don’t re-raise; let the model handle it semantically.
• “How do you avoid infinite loops in an agent?” — cap iterations (10-20 typical). Track whether the model is making progress (different tool calls, different args) — if not, break.
• “Parallel tool calls — when?” — when the model emits multiple tool_calls in one response (modern models do this). Execute them concurrently (asyncio.gather) for big latency wins.
• “What’s MCP?” — Model Context Protocol. Anthropic’s standard for tool/resource servers that AI models connect to. Portable across products; replaces bespoke per-model tool integration.
• “How does this compare to LangChain agents / ReAct?” — ReAct is a prompt technique where the model writes “Thought: … Action: … Observation: …” in free text. Function calling is the structured version — model emits a typed call, you execute, return result. Function calling is more reliable, easier to debug, and faster.