Tool contracts: designing APIs that models can actually call

Design the tool surface for the caller you have. A model cannot ask a clarifying question mid-call, cannot see your logs, and will retry a failed call verbatim unless the error tells it what to change.

When an agent misbehaves, the first instinct is to blame the model or edit the system prompt. In our experience the cause is more often the tool surface: an argument name that implies the wrong thing, an error that says only "invalid request", a tool that does two jobs, or a retry that silently created a second record.

Tool design is interface design. The user is a language model, and it has a specific profile: excellent at pattern-matching from names and descriptions, poor at holding invariants that are not written down, unable to experiment before committing, and prone to repeating an action that produced an ambiguous result.

Name and describe for the point of decision#

The model chooses a tool from its name and description alone, before seeing any result. That text is doing all the work.

# Weak — the model must guess what "process" means and when to use it
{"name": "process_data", "description": "Process the data"}

# Strong — states what it does, what it returns, when to use it, and the limits
{
  "name": "search_invoices",
  "description": (
    "Search invoices by customer, date range, or status. "
    "Returns up to 50 matching invoice IDs with amount and status — "
    "not line items. Use get_invoice for full detail on one invoice. "
    "Searches only the last 24 months; older records need request_archive."
  ),
}

The strong version answers the four questions a caller has at the decision point: what does it do, what comes back, what is the next tool if this is not the one, and what will it refuse to do. Every one of those, omitted, produces a characteristic failure: wrong tool chosen, result misinterpreted, dead end, or a confident query against data that does not exist.

If two tools' descriptions could both plausibly answer the same request, the model will pick between them at random. Overlap in a tool surface is not redundancy — it is nondeterminism.

Schemas are documentation the model cannot skip#

Constrained decoding means a well-specified schema is enforced, not merely suggested. Use that.

class SearchInvoices(BaseModel):
    customer_id: str = Field(pattern=r"^cus_[a-zA-Z0-9]{14}$",
                             description="From search_customers; not the display name")
    status: Literal["draft", "open", "paid", "void"] | None = None
    since: date | None = Field(None, description="Inclusive. Defaults to 90 days ago.")
    until: date | None = Field(None, description="Inclusive. Defaults to today.")
    limit: int = Field(20, ge=1, le=50)

Four things this does that prose cannot:

• The pattern on customer_id makes it structurally impossible to pass a company name — the single most common argument error we see in production agents.
• The enum on status removes an entire class of "activated" versus "active" guessing.
• The defaults mean the model does not have to invent values for parameters it has no opinion about. Every parameter without a default is a parameter the model might hallucinate.
• The bounds on limit prevent the "just fetch everything" strategy that blows up the context window.

Prefer flat schemas. Deeply nested objects are reliably harder for models to construct correctly, and a nested structure can nearly always be flattened or split into two calls.

Errors are the model's only feedback channel#

The model cannot read your logs, cannot inspect the database, and cannot ask you what happened. The error string is the entire debugging surface, and it is usually written as if a human on-call engineer will read it.

# Useless: the model has no idea what to change, so it retries verbatim
raise ValueError("Invalid request")

# Actionable: what was wrong, why, and exactly what to do next
raise ToolError(
    code="UNKNOWN_CUSTOMER",
    message=(
        "No customer with id 'cus_9fT2xQ1'. "
        "Customer ids come from search_customers. "
        "If you have a company name, call search_customers(name=...) first."
    ),
    retryable=False,
    suggested_tool="search_customers",
)

The retryable flag matters more than it looks. Without it, models retry everything — including deterministic failures — burning budget on calls that cannot succeed. Make retryability explicit and machine-readable, and enforce it in the runtime rather than hoping the model respects it.

A taxonomy that covers most of what a tool needs to express:

| Code class | Meaning | Model should | | --- | --- | --- | | INVALID_ARGUMENT | Malformed input | Fix the argument named in the message | | NOT_FOUND | Valid shape, no such entity | Look it up with the suggested tool | | PERMISSION_DENIED | Not allowed, ever | Stop; report to the user | | PRECONDITION_FAILED | Needs another step first | Do the prerequisite named | | RATE_LIMITED | Temporary | Back off; the runtime handles this | | UNAVAILABLE | Transient | Retry with backoff, bounded | | AMBIGUOUS | Multiple matches | Disambiguate using the returned candidates |

AMBIGUOUS is the one teams forget, and it prevents a genuinely bad outcome: returning the first of several matches and letting the agent act on the wrong record with full confidence. Return the candidates and force a choice.

Idempotency, because retries are guaranteed#

An agent will retry. Sometimes because the call timed out after the write succeeded, sometimes because it forgot it already called, sometimes because a supervisor re-ran a step. Any mutating tool without an idempotency story will eventually create duplicates.

async def create_invoice(args: CreateInvoice, ctx: ToolContext):
    # deterministic key from the semantic content of the request, plus the
    # run id — the same logical operation in the same run is the same operation
    key = sha256(f"{ctx.run_id}:{args.customer_id}:{args.amount}:{args.memo}")

    if existing := await store.get_by_key(key):
        return {**existing, "idempotent_replay": True}   # tell the model, honestly

    invoice = await billing.create(args)
    await store.put(key, invoice)
    return invoice

Returning idempotent_replay: True rather than silently succeeding is deliberate. The model learns it already did this, which is information it needs and would otherwise have to infer from a suspiciously familiar invoice id.

Granularity#

The most common structural mistake is tools that are too fine-grained. Twelve tools that must be called in sequence to accomplish one obvious operation means twelve opportunities to get the order wrong, and twelve round trips of latency.

The heuristic we use: a tool should correspond to a complete unit of user intent, not a database operation. book_appointment — which checks availability, holds the slot, creates the record, and sends confirmation — is one tool. Exposing those four steps separately means the agent can leave a slot held and never confirmed, which is a state your system now has to handle.

The counter-pressure is that coarse tools return large results and hide failures. Resolve it by making the tool coarse and the result structured: report which sub-steps succeeded, so a partial failure is legible.

Dry runs and confirmation#

For anything destructive or externally visible, a two-phase surface is worth the extra call:

class SendCampaign(BaseModel):
    segment_id: str
    template_id: str
    dry_run: bool = Field(True, description="Preview only. Set false to actually send.")

Defaulting dry_run to True means the model must make an affirmative choice to do the irreversible thing, and the preview result gives it — and any human in the loop — the recipient count before 40,000 emails leave the building. The pattern costs one extra call and prevents the category of incident that ends agent programmes.

Testing the surface#

Tool contracts are testable without any model in the loop, and then with one:

1. Schema fuzzing. Generate valid and invalid arguments; assert that every invalid one produces a specific, actionable error rather than a stack trace.
2. Description ambiguity check. For each pair of tools, ask a model which one answers a set of representative requests. Disagreement across runs means the descriptions overlap and need sharpening.
3. Retry safety. Call every mutating tool twice with identical arguments. Assert exactly one effect.
4. Error legibility. Take each error your tools can emit, give a model the error plus the tool list, and ask what it would do next. If the answer is "retry the same call", the message is not doing its job.

That third test catches more real bugs than anything else on the list, and it takes an afternoon to write.

The summary#

The tool surface is where most agent reliability is won or lost, and it is ordinary engineering: name things precisely, constrain what you can constrain, default what the caller has no opinion about, make errors actionable, make writes idempotent, and size tools to intent rather than to your schema.

Do that and a smaller model will outperform a larger one calling a bad surface.