Skip to content
AI.info

Generative AI

Function Calling and Tool Contracts

Design tool-calling interfaces with clear schemas, authorization, idempotency, result validation, and audit trails.

By the end you can

Visual

A safe tool loop has two authorities

The model chooses a proposal. The application decides whether and how it can run.

That split is not only a design preference. A system called CaMeL implements it as running code, in a paper first posted in March 2025 and revised that June. CaMeL extracts control and data flow from the trusted user query, so retrieved untrusted data can never alter the program that runs. It also attaches capabilities to values. The abstract says what that second half buys: “To further improve security, CaMeL uses a notion of a capability to prevent the exfiltration of private data over unauthorized data flows by enforcing security policies when tools are called.”

The separation has a price and the paper prints it. In the revised version, CaMeL solved 77% of AgentDojo tasks with provable security, against 84% for an undefended system. Seven points of utility, spent on a guarantee. The first version had reported 67% with provable security and no undefended baseline at all.

The execution end of the loop is specified rather than improvised. An IETF Standards Track draft defines an Idempotency-Key request header: a client-generated unique value that lets a server recognise a retry of a non-idempotent method such as POST or PATCH. It recommends a UUID. One line leaves no room: “The idempotency key MUST be unique and MUST NOT be reused with another request with a different request payload.” Stripe runs the same idea in production. It saves the status code and body of the first request under a given key and replays it on any repeat, a 500 included. Keys can run to 255 characters. They are pruned after at least 24 hours. Send the same key with different parameters and the API errors.

FigureProcess · 6 steps
  1. 1

    Describe available tools

    Expose narrow names, schemas, effects, and usage guidance.

  2. 2

    Generate a proposal

    The model selects a tool and produces candidate arguments.

  3. 3

    Validate arguments

    Code checks types, ranges, required fields, and cross-field invariants.

  4. 4

    Authorize the action

    Identity, role, consent, limits, and policy determine permission.

  5. 5

    Execute idempotently

    The tool handles retries, timeouts, and duplicate requests safely.

  6. 6

    Validate the result

    The application checks status, provenance, side effects, and next-step eligibility.

Comparison

Narrow tools are easier to authorize and evaluate

A single powerful tool shifts ambiguity into arguments and policy code. The narrower the declared effect, the less work the authorization layer has to do at the moment the call arrives. It is also less that a successful injection can accomplish with the credentials it borrows.

FigureComparison · 3 columns

Read-only query

Retrieve information without changing external state.

  • Lower consequence but privacy still matters
  • Supports grounding and clarification
  • Needs source and freshness validation
  • Can expose data through overbroad scope

Bounded mutation

Perform one reversible or tightly scoped change.

  • Clear arguments and limits
  • Often needs user confirmation
  • Supports idempotency keys
  • Easier to audit than general execution

General-purpose execution

Run arbitrary code, shell commands, or broad transactions.

  • Maximum flexibility
  • Large attack surface
  • Difficult permission model
  • Requires sandboxing and strong isolation

Tool descriptions shape proposals but do not enforce policy

Names, descriptions, enums, examples, and argument schemas help the model choose and populate a tool. Ambiguous schemas cause wrong-tool selection and invented values.

That failure has a name and a scoreboard. The first Berkeley Function Calling Leaderboard scored function calling on 2,000 question-function-answer pairs: 1,680 Python, 100 Java, 50 JavaScript, 70 REST API and 100 SQL. Each was graded twice, once by Abstract Syntax Tree matching and once by actually running the call. One category exists purely to catch the model that calls something anyway: “In function relevance detection, we design scenarios where none of the provided functions are relevant and supposed to be invoked.” There the graded correct output is no function call at all. A peer-reviewed version of the benchmark followed in 2025, much larger, carrying the same failure mode forward as the Irrelevance category alongside multi-turn and agentic tests.

Validation should reject unknown fields, unsupported combinations, and missing evidence. Security decisions must consult current application state, not the model’s explanation of why the action is allowed.

Case

A model trained to decide which API to call, and when

The choice itself can be learned. Toolformer, published in 2023, was trained to decide which APIs to call, when to call them, what arguments to pass and how to fold the results into future token prediction. The training signal was self-supervised. It required “nothing more than a handful of demonstrations for each API”. The tools were ordinary: a calculator, a Q&A system, a search engine, a translation system and a calendar.

The model is a 6.7B-parameter GPT-J finetuned to insert its own API calls, and it was measured against models an order of magnitude larger. On the LAMA question-answering subsets it scored 33.8 against GPT-3’s 26.8 on SQuAD, 11.5 against 7.0 on Google-RE, and 53.5 against 39.8 on T-REx. GPT-3 is the 175B model. The paper is blunt about it: “It also clearly outperforms OPT (66B) and GPT-3 (175B), despite both models being much larger.” On the math benchmarks Toolformer reached 40.4 on ASDiv, 29.4 on SVAMP and 44.0 on MAWPS, against GPT-3’s 14.0, 10.0 and 19.8.

The control condition is the part to keep. Disable the API calls and the same weights fall to 22.1 / 6.3 / 34.9 on the three LAMA subsets and 14.8 / 6.3 / 15.0 on the three math sets. ASDiv goes 14.8 → 40.4 for one reason only: the model is allowed to call the calculator. The gain lives in the execution the model does not itself perform. None of that changes who executes. A learned proposal is still a proposal, and the schema still has to be checked.

Key idea

Tool access turns prompt injection into an action problem

A retrieved page, email, or file can contain instructions that ask the model to disclose data or invoke a tool. The model may treat those instructions as relevant text.

AgentDojo turns that worry into a countable surface. Released in 2024, it is an evaluation environment for agents that call tools over untrusted data, and its abstract gives the inventory: “We populate the environment with 97 realistic tasks (e.g., managing an email client, navigating an e-banking website, or making travel bookings), 629 security test cases, and various attack and defense paradigms from the literature.” Every one of those 629 cases plants a prompt injection in content a tool returns. The authors also report the quieter half of the result. State-of-the-art LLMs fail many of the 97 ordinary tasks with no attack present at all.

Least privilege limits how far a successful one gets. Use allowlists, scoped credentials, sandboxing, data separation, user confirmation, and independent policy checks instead of relying on prompt wording to ignore malicious content.

Assume untrusted context can influence tool proposals; constrain what any proposal is allowed to accomplish.

Case

Instructions planted in the data the system was going to fetch

The attacker never touches the interface. That is the whole shape of indirect prompt injection, named and demonstrated in 2023. The application is compromised remotely “by strategically injecting prompts into data likely to be retrieved”, and the retrieval pipeline carries them in.

NIST reached the same place from the defensive side in March 2025. Its taxonomy of adversarial machine learning attacks states the cause plainly: “Because GenAI models combine the data and instruction channels, attackers can leverage the data channel to affect system operations”. The same section lists the three goals such an attack can serve: availability violation, integrity violation and privacy compromise.

Two years after the paper, the pattern arrived in a shipped product. CVE-2025-32711, known as EchoLeak, was published on 11 June 2025 against Microsoft 365 Copilot. An attacker-sent email carrying hidden instructions made Copilot exfiltrate data from the victim’s tenant, with no user action at all. Microsoft, as the assigning CNA, describes it in one line: “Ai command injection in M365 Copilot allows an unauthorized attacker to disclose information over a network.” It is classified CWE-74, improper neutralization of special elements in output used by a downstream component. Microsoft scored it CVSS 3.1 9.3 Critical; the NVD record’s primary CVSS 3.1 metric is 7.5 High. Aim Labs reported it, and it was fixed server-side with no customer action required. No prompt-level instruction the users could have written would have mattered either way.

Analogy

A junior operator can draft a work order without holding the master key

On the shop floor the junior operator writes the work order and a supervisor signs it, checking identity, scope, safety, and duplicate requests. Only then does the machinery change state. Function calling wants the same separation between proposal and authority. What the shop never has to consider is a work order arriving ten thousand times an hour, or an operator who follows an instruction somebody slipped into the paperwork. A tool-enabled model faces both.

Both have answers, and neither answer is a better-worded instruction to the operator. The duplicate is handled by a key the caller generates and the server remembers, unique per payload, replayed rather than re-executed. The planted instruction is what CVE-2025-32711 was, and what CaMeL’s capabilities are for. The paperwork stays data. The signature stays with the supervisor.

Give the model expressive proposal power and keep execution authority deterministic, narrow, and observable.

Steps

Test a tool workflow as a transaction system

Include misuse, ambiguity, failure, and recovery cases.

τ-bench shows what that standard costs when someone actually applies it. Published in 2025, it runs a tool-calling agent against a simulated user in two domains, retail and airline. The agent gets domain API tools and a written policy. Grading ignores the transcript and compares the final database state to an annotated goal state. The result: “Our experiments show that even state-of-the-art function calling agents (like gpt-4o) succeed on <50% of the tasks, and are quite inconsistent (pass^8 <25% in retail).” pass^k is the paper’s own metric, the fraction of tasks solved in all k independent trials. Under 50% once. Under 25% across eight independent runs of the same task. An agent that passes a demo is not an agent that passes a shift, and only the repeated trial tells you which one you have.

FigureProcess · 6 steps

  1. 1. Enumerate allowed effects

    Document state changes, data exposure, cost, and reversibility.

  2. 2. Validate argument boundaries

    Use invalid IDs, stale state, extreme values, and conflicting fields.

  3. 3. Exercise permissions

    Test users, roles, tenants, revocation, and cross-account attempts.

  4. 4. Simulate retries

    Verify idempotency under timeout and duplicate delivery.

  5. 5. Inject hostile context

    Place malicious instructions in retrieved and tool-returned content.

  6. 6. Inspect the trace

    Confirm that proposals, checks, confirmations, execution, and results are reconstructible.

Tool use crosses from language into consequence

Function calling turns model output into a structured interface with external systems. The reliability standard therefore has to include transaction semantics, permissions, and incident recovery. The numbers in this lesson are all measurements of that crossing: 14.8 → 40.4 on ASDiv when the calculator is reachable, pass^8 below 25% in τ-bench retail, 629 planted injections in AgentDojo, 77% against 84% as the priced cost of CaMeL’s guarantee, and one CVE where the whole exploit was an email.

Conversation state comes next. The application must decide what information persists across calls and which memory claims the model is allowed to rely on.

Key takeaways