Tool Calling: How a Model Picks a Function

"The model called our API" is a convenient shorthand and a misleading one. The model produced a block of JSON. Something in your code read that JSON and decided to make a call. Everything that matters about tool calling lives in that gap.

What you actually send

A tool definition has three parts, and they are not equally important:

{
  name: "search_orders",
  description: "Find orders for a customer.
    Use when the user asks about an order's
    status, contents, or delivery date.",
  input_schema: {
    type: "object",
    properties: {
      customer_id: { type: "string" },
      status: { enum: ["open","shipped"] },
      limit: { type: "integer" }
    },
    required: ["customer_id"],
    additionalProperties: false
  }
}

The schema constrains the shape of the arguments. The description is what decides whether the tool gets picked at all, and for what. It is prompt text — sent on every request, read every turn, and competing for attention with every other tool's description.

const tools: Anthropic.Tool[] = [
  {
    name: 'search_orders',
    description:
      'Find orders for a customer. Use when the user asks about an order\'s ' +
      'status, contents, or delivery date. Returns at most 20 orders, newest ' +
      'first. Does not include cancelled orders unless status is set.',
    input_schema: {
      type: 'object',
      properties: {
        customer_id: { type: 'string', description: 'Internal id, format c_XXXX' },
        status: { type: 'string', enum: ['open', 'shipped'] },
        limit: { type: 'integer', minimum: 1, maximum: 20 },
      },
      required: ['customer_id'],
      additionalProperties: false,
    },
  },
]

Note what the description carries that the schema cannot: when to use it, what it returns, and what it excludes. Those sentences prevent more bad calls than any amount of schema tightening.

Descriptions are an interface for a reader, not a compiler

The failure mode is writing descriptions the way you write code comments — restating the function name. "Searches orders" tells a model nothing it couldn't guess.

Useful descriptions answer questions the model will otherwise guess at:

• When should this be used, and when not? "Use for order history. For live delivery tracking use track_shipment instead."
• What are the arguments, really? "customer_id is the internal id (c_8812), not the email address."
• What comes back? "Returns a list, possibly empty. An empty list means no orders, not an error."
• What are the consequences? "This issues a refund immediately and cannot be undone."

That last one matters more than it looks. A model that knows an action is irreversible behaves noticeably more conservatively about calling it. It's also cheap to add: one sentence per tool, written once.

Validate, always

With strict mode (strict: true on the tool definition, plus additionalProperties: false and a complete required list), the API guarantees the arguments validate against your schema. Without it, they usually do — and "usually" is not a property you can build on.

Either way, validate at the boundary:

const parsed = SearchOrders.safeParse(block.input)   // zod, ajv, whatever

if (!parsed.success) {
  return {
    type: 'tool_result',
    tool_use_id: block.id,
    is_error: true,
    content: `Invalid arguments: ${parsed.error.message}`,
  }
}

Two reasons. First, schema validity is not semantic validity — a well-formed customer_id can still be a customer that does not exist. Second, the arguments are model output flowing into your systems. Treat them the way you treat a request body from the internet: authorize the user the agent is acting for, never the model's assertion about who it is acting for.

Error messages are repair instructions

This is the highest-leverage thing in tool calling and the most neglected. The error string you return is read by the model and is the entire basis for its next attempt.

✗  "Invalid input"
✗  "Error: 422"
✓  "status must be one of: open, shipped. Got: pending.
    For cancelled orders use include_cancelled: true."

The third one gets fixed on the next turn. The first two produce either the same call again or a confused apology to the user. Write tool errors the way you would write a compiler error: what was wrong, what was expected, what to do instead.

Fewer tools, and coarser

Every tool definition costs tokens on every request and competes for the model's attention. The reliability curve bends the wrong way well before you think it will.

When two tools overlap, the model has to guess, and a guess is a coin flip you pay for. get_user, get_user_by_email, and lookup_customer should be one tool with an argument. If your surface is genuinely large — a big MCP server, a broad internal API — the answer is not a longer list; it is giving the model a way to find tools rather than listing them all.

Parallel calls, and the trap in handling them

One assistant message can contain several tool_use blocks. Run them concurrently, then return every result in a single user message:

const calls = response.content.filter((b) => b.type === 'tool_use')
const results = await Promise.all(calls.map(runTool))
messages.push({ role: 'user', content: results })   // all of them, one message

Splitting results across several user messages is the trap: it breaks the pairing between tool_use and tool_result blocks and trains the model out of batching, which makes every subsequent turn slower.

Controlling whether a tool is called

By default the model decides whether to use a tool at all. The request's tool_choice parameter changes that:

You can also disable parallel tool use when your tools must run strictly one at a time — say, because each depends on state the previous one changed.

A tool as an output format

Forcing a specific tool is a reliable way to get structured data out of a model even when nothing is "called":

const response = await client.messages.create({
  model, max_tokens: 1024,
  tools: [{
    name: 'record_ticket_triage',
    description: 'Record the triage decision for a support ticket.',
    input_schema: {
      type: 'object',
      properties: {
        category: { type: 'string', enum: ['billing', 'bug', 'account', 'other'] },
        priority: { type: 'string', enum: ['low', 'normal', 'urgent'] },
        summary: { type: 'string', description: 'One sentence, for the queue view' },
      },
      required: ['category', 'priority', 'summary'],
      additionalProperties: false,
    },
  }],
  tool_choice: { type: 'tool', name: 'record_ticket_triage' },
  messages: [{ role: 'user', content: ticketText }],
})

const triage = response.content.find((b) => b.type === 'tool_use')!.input

The "tool" never runs; its input is the result. This predates dedicated structured-output features and still works well — see structured outputs from LLMs for the trade-offs between the approaches.

What a good tool result looks like

The result is prompt input for the next turn, and it's re-sent on every turn after that. Design it as carefully as the description:

• Small and relevant. Return the fields the model needs to act, not the whole API response. A 40-field order object where the question was "has it shipped?" costs tokens and attention for twenty more turns.
• Readable. Short JSON or plain text with labels both work; deeply nested JSON with opaque keys (st: 3) forces the model to guess. Use the words a person would use.
• Explicit about emptiness and truncation. "No orders found for c_8812" and "Showing 20 of 312 orders; pass cursor to see more" both prevent the model inventing a conclusion.
• Self-contained IDs. If the model will need to refer to an item in a later call, include the exact ID it should pass back.

Timeouts and side effects

Your tool handlers are ordinary backend code running on the model's behalf, so the ordinary rules apply:

• Give every tool a timeout, and return a timeout as an error result ("The search took too long; try a narrower query") rather than letting the whole loop hang.
• Make write tools idempotent. The harness may retry a tool after a network failure, and the model may call the same tool twice. Pass an idempotency key derived from the tool_use ID, so a repeated call is harmless.
• Separate reads from writes. Read tools can usually run freely and in parallel; write tools deserve confirmation, rate limits and audit logs. Naming them so the difference is obvious helps both the model and your policy layer.

When the tool surface is large

Past a few dozen tools, a flat list stops working. Three patterns:

• Group into fewer, broader tools with an action argument — one github tool with action: "search_issues" | "get_pr" | … — at the cost of a more complex schema.
• Tool search: give the model a find_tools(query) tool that returns the definitions of the few relevant tools, and add those to the request. The model starts each task with a small surface and loads what it needs.
• Scope by task: decide in code which tools a request type needs, and send only those. A billing question doesn't need the deployment tools.

Measuring tool selection

"Does the model pick the right tool?" is an empirical question. Build a small set of realistic user requests, each labelled with the tool (and key arguments) a good agent should call first, and run it whenever you change a description, add a tool or switch models:

for (const c of cases) {
  const res = await client.messages.create({ model, tools, max_tokens: 512,
    messages: [{ role: 'user', content: c.input }] })
  const call = res.content.find((b) => b.type === 'tool_use')
  record(c.id, call?.name === c.expectedTool)
}

Fifty cases catch most description regressions. The confusions it surfaces — two tools the model keeps mixing up — usually point to a description to sharpen or two tools to merge. See evals for LLM apps.

A description, before and after

A real pattern: an agent keeps calling get_customer with an email address, the lookup fails, and it apologises to the user. The tool:

name: get_customer
description: Gets a customer.
input: { id: string }

Nothing tells the model id isn't an email. The fix is almost entirely prose:

name: get_customer
description: Look up one customer by internal id (format c_XXXX) or by
  email address. Returns name, plan, signup date and account status.
  Use this before any billing or order tool, which need the internal id.
  Returns an error if no customer matches; it never guesses.
input: { id?: string (c_XXXX), email?: string } — provide exactly one

Three changes did the work: the tool now accepts what users actually give (an email), the description says what comes back and which tools need its output, and the schema says the two inputs are alternatives. Run the selection eval before and after a change like this; it's the fastest feedback loop available for agent quality.

Approval gates belong in the harness

The model asking to do something and your code doing it are separate events, which is exactly where a human fits:

if (DESTRUCTIVE.has(block.name)) {
  const ok = await requestApproval(block)      // ask a person
  if (!ok) {
    return {
      type: 'tool_result',
      tool_use_id: block.id,
      is_error: true,
      content: 'The user declined this action. Do not retry it.',
    }
  }
}

Note the denial is phrased as an instruction, not just a status. "Denied" invites a retry; "the user declined, do not retry" ends it. Log the decision either way — who approved what, and when — because for destructive actions the audit trail matters as much as the gate.

The summary

• The description decides whether a tool is used correctly; the schema only decides whether the arguments parse.
• Validate the arguments, and authorize the user — not the model.
• Error strings are prompt input. Make them say how to fix the problem.
• Fewer, coarser, non-overlapping tools beat a complete API surface.
• Return every result, in one message, including the failures.
• Keep results small, explicit about truncation, and carrying the IDs the model will need next.
• Measure tool selection with a labelled set of requests, and rerun it on every description change.