Function calling in LLMs: how tool calls work, end to end
How LLM tool calling works: the schema, the call the model emits, your code running it, the result going back. Claude and OpenAI shapes, plus a tested loop.
Function calling (most providers now say "tool calling") lets a model ask your program to run a function. The model never runs anything. It reads a list of tools you describe, decides one would help, and replies with the tool's name and a JSON object of arguments. Your code runs the function, sends the result back, and the model writes its answer from that result.
That is four steps, and most bugs live in the seams between them: a schema the model misreads, arguments that do not match it, a result attached to the wrong call, or a loop that never ends. This post walks the full round trip, shows the current request and response shapes for the Anthropic Messages API and the OpenAI Responses API (checked against their docs in November 2026), and ends with a loop you can run without an API key, plus tests for the history rules that cause most 400 errors.
The round trip#
your app model API
-------- ---------
1. send: user message + tool schemas --->
2. model decides a tool helps
<--- returns tool call(s): name + JSON args + id
3. validate args, run the function
4. send: tool result(s) tagged with --->
the same id
5. model reads the result
<--- final text (or more tool calls: go to 3)
The model sees only what you put in the request. It cannot reach your database or the weather service; it can only ask. That split matters for safety too. Every side effect happens in code you wrote, so that code is where permission checks, rate limits and logging belong.
Step 5 can loop. A model asked "is Delhi warmer than Bengaluru?" may call the weather tool twice in one turn, get the results, and answer. A model asked to book a meeting may call find_free_slots, then create_event, then answer. Each round is one more model call, with the full history resent each time, so the token bill grows with the number of rounds.
Step 1: describe the tool#
A tool definition has three parts: a name, a description, and a JSON Schema for the arguments. Both providers use JSON Schema for the argument shape. They differ in the wrapper.
Anthropic puts the schema under input_schema (define tools):
{
"name": "get_weather",
"description": "Current temperature for one city. Call once per city.",
"input_schema": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City name, e.g. Delhi"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
},
"required": ["city", "unit"],
"additionalProperties": false
}
}
OpenAI's Responses API uses a flat object with type: "function" and the schema under parameters (function calling guide):
{
"type": "function",
"name": "get_weather",
"description": "Current temperature for one city. Call once per city.",
"parameters": { "...same schema as above...": true },
"strict": true
}
The older Chat Completions API nests the same fields one level deeper, under a function key. If you copy an example from 2023, that nesting is the usual reason it breaks on the Responses API.
The description does more work than the schema. Anthropic's docs ask for a "detailed plain text description of what the tool does, when it should be used, and how it behaves." The model picks tools by reading these descriptions, so "Call once per city" in the example above is an instruction the model will follow. A vague description like "weather tool" leads to wrong picks once you have ten tools that overlap.
Two schema habits help. Use enum wherever the set of values is closed, because a free-text unit field will eventually receive "C", "Celsius" and "metric". And leave out arguments your app already knows. OpenAI's guide makes this point with a refund example: if the order id is already in your session, do not make the model fill it in.
Step 2: the model emits a call#
With Anthropic, a response that wants a tool has stop_reason: "tool_use" and one or more tool_use content blocks (handle tool calls). Each block has an id, a name, and an input that is already a parsed object:
{
"stop_reason": "tool_use",
"content": [
{"type": "text", "text": "Checking both cities."},
{"type": "tool_use", "id": "toolu_01", "name": "get_weather",
"input": {"city": "Delhi", "unit": "celsius"}},
{"type": "tool_use", "id": "toolu_02", "name": "get_weather",
"input": {"city": "Bengaluru", "unit": "celsius"}}
]
}
With the OpenAI Responses API, the output array holds items of type: "function_call", each with a call_id, a name, and arguments as a JSON-encoded string:
{"type": "function_call", "call_id": "call_12345xyz", "name": "get_weather",
"arguments": "{\"city\":\"Delhi\",\"unit\":\"celsius\"}"}
That string is a real difference. With OpenAI you call json.loads yourself, and without strict mode a malformed string is possible, so the parse needs a try.
Step 3: run it, and validate first#
Your code looks up the function by name, checks the arguments against the schema, and runs it. Validation is not optional, even with strict mode on. Strict mode guarantees the arguments match the schema's shape. It does not guarantee that "Pune" exists in your data, that the date is in the future, or that the user is allowed to see that customer. Anthropic's strict mode also does not enforce numerical constraints such as minimum and maximum (JSON Schema limitations), so a field with minimum: 1 can still arrive as 0 unless you check it.
Step 4: send the result back#
The result has to name the call it answers. With Anthropic, that is a tool_result block in the next user message, with tool_use_id set to the block's id:
{"role": "user", "content": [
{"type": "tool_result", "tool_use_id": "toolu_01",
"content": "{\"city\": \"Delhi\", \"temp\": 31.0, \"unit\": \"celsius\"}"},
{"type": "tool_result", "tool_use_id": "toolu_02",
"content": "{\"city\": \"Bengaluru\", \"temp\": 24.5, \"unit\": \"celsius\"}"}
]}
Anthropic's docs set two ordering rules. The tool results must come in the message right after the assistant message that asked for them, with nothing in between. Inside that user message, the tool_result blocks come first and any text comes after. Break either rule and you get an error such as "tool_use ids were found without tool_result blocks immediately after."
With OpenAI Responses, you append the model's function_call items to your input list, then add one item per call:
{"type": "function_call_output", "call_id": "call_12345xyz",
"output": "{\"city\": \"Delhi\", \"temp\": 31.0, \"unit\": \"celsius\"}"}
OpenAI's guide adds a rule for reasoning models such as GPT-5: any reasoning items that came back alongside the tool calls must also be passed back with the outputs. Chat Completions uses a different shape again, a message with role: "tool" and a tool_call_id.
Parallel calls#
Both APIs let the model ask for several tools in one turn, as in the two-city example. Anthropic's parallel tool use page states the rule that keeps this working: every result goes back in a single user message, with no text before the results. Sending one user message per result is a common bug. The API may accept it, but the model learns from the history that it should stop making parallel calls.
If your tools have side effects that must happen in order, turn parallel calls off. Anthropic uses disable_parallel_tool_use: true inside tool_choice. OpenAI uses a top-level parallel_tool_calls: false, which its docs say "ensures exactly zero or one tool is called."
You are free to run parallel calls concurrently in your own code (threads, asyncio.gather). The model does not care how you ran them, only that each id gets exactly one result.
Strict schemas#
Without strict mode, schema adherence is best effort. The model usually gets it right, and sometimes sends "2" for an integer or skips a required field. Strict mode constrains the model's sampling so the arguments always parse and match the schema.
Anthropic: add "strict": true to the tool definition (strict tool use). Only a subset of JSON Schema is supported; numeric bounds, for example, are not.
OpenAI: add "strict": true to the function. Two requirements apply: additionalProperties must be false on every object, and every field in properties must be listed in required. To make a field optional, give it a type like ["string", "null"]. If you send strict: true with a schema that breaks these rules, the request is rejected. If you leave strict out, the Responses API tries to convert your schema to strict mode and falls back to non-strict if it cannot; Chat Completions stays non-strict by default.
The "every field required" rule trips people porting a schema from Anthropic to OpenAI. A Claude tool with an optional country field becomes a 400 on OpenAI strict mode until you add country to required and allow null.
Tool-choice settings#
By default the model decides whether to call a tool. You can change that.
| Goal | Anthropic tool_choice |
OpenAI tool_choice |
|---|---|---|
| Model decides (default with tools) | {"type": "auto"} |
"auto" |
| Must call some tool | {"type": "any"} |
"required" |
| Must call this tool | {"type": "tool", "name": "get_weather"} |
{"type": "function", "name": "get_weather"} |
| No tool calls | {"type": "none"} |
"none" |
| Limit to a subset | (send fewer tools) | {"type": "allowed_tools", ...} |
There is a catch on the Anthropic side that is easy to miss. The define-tools page says forced tool use (any or tool) returns a 400 error on Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and 5.2, and Mythos 5.1 and 5.2. It also fails when manual extended thinking is on. The docs' suggested replacement is auto plus strict tool use, or structured outputs when what you want is a fixed JSON reply rather than an action. On models that do support forcing, a forced call starts the response with the tool call, so you get no explanatory text first. Changing tool_choice between requests also invalidates the message part of Anthropic's prompt cache.
If you forced a tool only to get JSON out of the model, structured outputs are the better tool for that job on both platforms.
Errors are results too#
When a tool fails, do not raise out of the loop. Send the failure back as a result so the model can recover. Anthropic has an explicit flag: "is_error": true on the tool_result. The docs warn that the model may call the tool again after an error, so cap retries. OpenAI has no error flag; you put the error text in output and the model reads it.
Write error messages for the model, not for a log file. "invalid arguments: 'kelvin' is not one of ['celsius', 'fahrenheit']" tells it exactly what to change. "Error 500" tells it nothing, and it will often retry the same call.
A runnable loop with a stubbed model#
The loop below has no API key and no network. The model is a small class that replays scripted responses in the Anthropic shape; the comment in StubModel marks where a real client.messages.create(...) call goes. The weather data is a dictionary. Everything else (validation with jsonschema 4.26.0, error results, the turn cap, the one-message rule for parallel results) is what a production loop needs.
import json
from jsonschema import Draft202012Validator
# ---- 1. Tools: a schema the model sees, a function your app runs ----
WEATHER = {"delhi": 31.0, "bengaluru": 24.5, "london": 9.0} # fake data, no network
def get_weather(city, unit):
c = WEATHER.get(city.lower())
if c is None:
raise LookupError(f"no data for city '{city}'")
temp = c if unit == "celsius" else round(c * 9 / 5 + 32, 1)
return {"city": city, "temp": temp, "unit": unit}
TOOLS = {
"get_weather": {
"fn": get_weather,
"schema": {
"name": "get_weather",
"description": "Current temperature for one city. Call once per city.",
"input_schema": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City name, e.g. Delhi"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["city", "unit"],
"additionalProperties": False,
},
},
}
}
# ---- 2. The model, behind an interface. Swap StubModel for a real client. ----
class StubModel:
"""Replays scripted responses in the Anthropic Messages shape.
Real version: client.messages.create(model=..., max_tokens=..., tools=tools, messages=messages)
"""
def __init__(self, script):
self.script = list(script)
self.calls = 0
def create(self, messages, tools):
self.calls += 1
return self.script.pop(0)
def tool_use(id_, name, inp):
return {"type": "tool_use", "id": id_, "name": name, "input": inp}
# ---- 3. Execute one call: validate, run, turn any failure into an error result ----
def run_tool(block):
tool = TOOLS.get(block["name"])
if tool is None:
return {"type": "tool_result", "tool_use_id": block["id"],
"content": f"unknown tool '{block['name']}'", "is_error": True}
errors = [e.message for e in Draft202012Validator(tool["schema"]["input_schema"]).iter_errors(block["input"])]
if errors:
return {"type": "tool_result", "tool_use_id": block["id"],
"content": "invalid arguments: " + "; ".join(errors), "is_error": True}
try:
out = tool["fn"](**block["input"])
except Exception as exc: # the model reads this and can recover
return {"type": "tool_result", "tool_use_id": block["id"],
"content": f"{type(exc).__name__}: {exc}", "is_error": True}
return {"type": "tool_result", "tool_use_id": block["id"], "content": json.dumps(out)}
# ---- 4. The loop ----
def run(model, user_text, max_turns=5):
messages = [{"role": "user", "content": user_text}]
tools = [t["schema"] for t in TOOLS.values()]
for turn in range(max_turns):
resp = model.create(messages, tools)
messages.append({"role": "assistant", "content": resp["content"]})
if resp["stop_reason"] != "tool_use":
return resp["content"][0]["text"], messages
calls = [b for b in resp["content"] if b["type"] == "tool_use"]
results = [run_tool(b) for b in calls] # every call gets a result
for r in results:
print(f" turn {turn}: {r['tool_use_id']} -> {r['content']}"
+ (" [is_error]" if r.get("is_error") else ""))
messages.append({"role": "user", "content": results}) # all in ONE message
raise RuntimeError(f"gave up after {max_turns} model calls")
# ---- 5. A scripted run: parallel calls, one bad argument, a retry, a final answer ----
script = [
{"stop_reason": "tool_use", "content": [
{"type": "text", "text": "Checking both cities."},
tool_use("toolu_01", "get_weather", {"city": "Delhi", "unit": "celsius"}),
tool_use("toolu_02", "get_weather", {"city": "Bengaluru", "unit": "kelvin"}),
]},
{"stop_reason": "tool_use", "content": [
tool_use("toolu_03", "get_weather", {"city": "Bengaluru", "unit": "celsius"}),
]},
{"stop_reason": "end_turn", "content": [
{"type": "text", "text": "Delhi is 31.0 C and Bengaluru is 24.5 C, so Delhi is 6.5 C warmer."},
]},
]
if __name__ == "__main__":
model = StubModel(script)
answer, history = run(model, "Is Delhi warmer than Bengaluru right now?")
print("answer:", answer)
print("model calls:", model.calls, "| messages in history:", len(history))
Output on Python 3.15:
turn 0: toolu_01 -> {"city": "Delhi", "temp": 31.0, "unit": "celsius"}
turn 0: toolu_02 -> invalid arguments: 'kelvin' is not one of ['celsius', 'fahrenheit'] [is_error]
turn 1: toolu_03 -> {"city": "Bengaluru", "temp": 24.5, "unit": "celsius"}
answer: Delhi is 31.0 C and Bengaluru is 24.5 C, so Delhi is 6.5 C warmer.
model calls: 3 | messages in history: 6
The script plays out a realistic failure. The model asks for two cities in parallel, gets one argument wrong, reads the error, and fixes it on the next turn. The good Delhi result is not thrown away when the Bengaluru call fails, because each call gets its own result. That cost three model calls instead of two, and the third call resent the whole six-message history. Strict mode would have stopped "kelvin" before it left the model, which is the practical case for turning it on.
To make this real, replace StubModel.create with a call to the Anthropic SDK, and replace resp["content"] with the SDK's response blocks (block.type, block.id, block.input are attributes there, not dictionary keys).
The OpenAI side#
The same run_tool works for OpenAI if you convert the shapes at the edge. The extra step is parsing the arguments string:
import json
from loop import run_tool
def run_openai_calls(output_items):
"""Responses API: function_call items in, function_call_output items out."""
outputs = []
for item in output_items:
if item["type"] != "function_call":
continue
try:
args = json.loads(item["arguments"]) # arguments arrive as a JSON string
except json.JSONDecodeError as exc:
outputs.append({"type": "function_call_output", "call_id": item["call_id"],
"output": f"arguments were not valid JSON: {exc}"})
continue
r = run_tool({"id": item["call_id"], "name": item["name"], "input": args})
outputs.append({"type": "function_call_output", "call_id": item["call_id"],
"output": r["content"]})
return outputs
fake_output = [
{"type": "function_call", "call_id": "call_a", "name": "get_weather",
"arguments": "{\"city\":\"London\",\"unit\":\"fahrenheit\"}"},
{"type": "function_call", "call_id": "call_b", "name": "get_weather",
"arguments": "{\"city\":\"Delhi\",\"unit\":"},
]
for o in run_openai_calls(fake_output):
print(o)
{'type': 'function_call_output', 'call_id': 'call_a', 'output': '{"city": "London", "temp": 48.2, "unit": "fahrenheit"}'}
{'type': 'function_call_output', 'call_id': 'call_b', 'output': 'arguments were not valid JSON: Expecting value: line 1 column 24 (char 23)'}
London's 9.0 C comes back as 48.2 F (9 x 9/5 + 32). The truncated second call still gets an output, so the model sees why it failed instead of a missing id.
Test the history, not only the tools#
Most tool-calling errors in production are history errors: a result missing for one id, a result in the wrong message, text placed before the results. These are cheap to test without a model. The four tests below run against the stub in 0.02 seconds:
from loop import StubModel, run, run_tool, script, tool_use
def check_history(messages):
for prev, msg in zip(messages, messages[1:]):
if prev["role"] != "assistant":
continue
ids = {b["id"] for b in prev["content"] if b["type"] == "tool_use"}
if not ids:
continue
assert msg["role"] == "user", "tool results must come right after the tool_use turn"
got = [b["tool_use_id"] for b in msg["content"] if b["type"] == "tool_result"]
assert set(got) == ids, f"missing results for {ids - set(got)}"
assert msg["content"][0]["type"] == "tool_result", "results must come first"
def test_happy_path_history_is_valid():
answer, history = run(StubModel(list(script)), "Is Delhi warmer?")
assert "6.5" in answer
check_history(history)
def test_bad_enum_becomes_error_result():
r = run_tool(tool_use("t1", "get_weather", {"city": "Delhi", "unit": "kelvin"}))
assert r["is_error"] and "kelvin" in r["content"]
def test_unknown_city_and_unknown_tool():
assert run_tool(tool_use("t2", "get_weather", {"city": "Pune", "unit": "celsius"}))["is_error"]
assert run_tool(tool_use("t3", "get_stock", {}))["is_error"]
def test_loop_stops_when_model_never_finishes():
stuck = [{"stop_reason": "tool_use", "content": [
tool_use(f"t{i}", "get_weather", {"city": "Pune", "unit": "celsius"})]} for i in range(10)]
try:
run(StubModel(stuck), "weather in Pune?", max_turns=3)
except RuntimeError as e:
assert "3" in str(e)
else:
raise AssertionError("loop should have stopped")
.... [100%]
4 passed in 0.02s
check_history encodes Anthropic's two ordering rules, so you can run it on logged conversations from a real model too. Whether the model picks the right tool with the right arguments is a separate question that needs a real model and a set of test prompts; evaluating an AI agent in Python builds that harness.
Common bugs#
- Results split across messages. One user message per parallel result. Collect them and send one message.
- A call with no result. An exception in one tool skips the result for that id, and the next request fails. Wrap each tool call so every id gets a result, as
run_tooldoes. - Text before the results. "Here are the results:" followed by
tool_resultblocks breaks Anthropic's ordering rule. - Not parsing OpenAI arguments.
item["arguments"]["city"]fails becauseargumentsis a string. - Dropping reasoning items. With OpenAI reasoning models, leaving them out of the next request when you send tool outputs.
- Forcing a tool on a model that rejects it.
tool_choice: {"type": "any"}on Claude Opus 5.5 is a 400. - No turn cap. A model that keeps getting errors keeps calling.
max_turnsturns an endless bill into an exception you can see. - Trusting arguments. The model chose
customer_id=1043from text a user typed. Check permissions in your code, not in the description. - Too many similar tools. Thirty tools with overlapping descriptions raise the wrong-pick rate. Merge them, or send only the tools that fit the current step.
Where this goes next#
Everything above is in-process: your tools are Python functions in the same program as the loop. When you want tools that live in another process and work with any client (Claude Desktop, an IDE, your own script), that is what MCP standardises. MCP from zero builds a server and a client, and the client ends up running this same loop with tool schemas it discovered over the protocol. For how MCP relates to agent-to-agent protocols, see MCP vs A2A vs ANP. For the agent patterns built on top of this loop (ReAct, planners, multi-agent), see AI agent architectures explained.
Sources#
- Anthropic: Define tools (tool definition fields,
tool_choiceoptions, forced tool use restrictions) - Anthropic: Handle tool calls (
tool_useandtool_resultshapes, ordering rules,is_error) - Anthropic: Parallel tool use
- Anthropic: Tool use overview (
disable_parallel_tool_use) - Anthropic: Strict tool use
- Anthropic: Structured outputs, JSON Schema limitations
- OpenAI: Function calling guide (Responses shapes,
tool_choice,parallel_tool_calls, strict mode) - OpenAI: Chat Completions create reference (
tool_call_id) - JSON Schema
- jsonschema on PyPI