Function calling

Letting an LLM call functions/tools in your code. The model outputs structured JSON (function name + arguments); your code executes it and returns the result. Enables LLMs to interact with the real world — APIs, databases, code execution. The basis of AI agents.

When to use

When the LLM needs to act on the world (APIs, DBs, code).
Structured data extraction.
Agent workflows.
Any LLM app needing external data or actions.

When not to

Simple Q&A (no tools needed).
When a single prompt suffices.
When you need guaranteed correctness (validate all calls).

Definition

Function calling (or tool use) lets an LLM output structured calls to functions you define. You describe available functions; the model picks one and outputs the arguments as JSON. Your code runs the function and returns the result to the model, which continues the conversation.

Why it exists

LLMs only output text — they can't call APIs or run code. Function calling bridges the gap: the model decides what to call, your code executes it, and the model uses the result.

How it works

1. Define tools (name, description, parameters JSON schema). 2. Send to LLM with user query. 3. Model outputs tool_call with args. 4. Your code executes the function. 5. Return result to model. 6. Model generates final response. Repeat for multi-step.

Scaling limits

The model decides which tool to call — it can pick wrong or hallucinate args. Multiple round trips add latency. Security: validate args before executing. Not all models support it equally well.

Cost shape

Each tool call adds LLM tokens (cost). Multiple calls per task = expensive.

Alternatives

MCP — Standardized, more infrastructure.
LangChain tools — Framework-managed, more abstraction.
Structured output (JSON mode) — No execution, just formatting.

Failure modes

Hallucinated function names/args.
Wrong tool selection.
Security (validate all inputs).
Latency from multiple round trips.

LLM (Large Language Model), AI agent, MCP (Model Context Protocol)