Designing a Tool Surface
Three well-shaped tools beat thirty CRUD endpoints. Every tool you add costs schema tokens on every turn and gives the model one more thing to choose wrongly; every tool you leave out is work the model has to assemble from parts, one model call per part, with a chance of stopping halfway. The surface is where those two pressures meet, and it is a design artefact rather than an export of your API index.
Sundry's settled at nine. It started at the four read-only ones, was briefly twenty-two, and lost two along the way. How it got there is more useful than the final list, because the same forces act on every agent that touches a real system with a decade of endpoints behind it.
Granularity
Tools should match the task, not the database table. start_return generates the shipping label, books the courier pickup and writes the return record, because at Sundry those three always happen together and nothing sensible exists in between them. Split into create_label, book_pickup and record_return, the same work costs three model calls instead of one, and it introduces a state that has never existed in the business: a label with no pickup, created because the run stopped on turn 9.
The rule that falls out of it: if two steps are always sequential and the intermediate state is not a state the business recognizes, they are one tool. The moment you split them you have made the model an orchestrator of your internals, and orchestration is exactly the job your existing, tested, deterministic code already does better. Every seam you expose is a seam a probabilistic component can stop at, which is Chapter 8's whole subject.
Selection Accuracy Falls With Size
Accuracy per tool declines as the list grows, and it declines fastest when neighbours overlap. Vera measured three versions of the Sundry surface on the same labelled tickets: the read-only core, the nine she shipped, and a twenty-two-tool version generated from the orders service's OpenAPI document.
| Surface | Tools | Correct first call | Runs with a wrong call | Model calls per ticket |
|---|---|---|---|---|
| Read-only core | 4 | 97% | 6% | 3.4 |
| The shipped surface | 9 | 93% | 14% | 4.1 |
| Generated from the API | 22 | 76% | 41% | 6.8 |
The middle column is the headline and the last one is the bill. Going from nine tools to twenty-two did not add capability — the same work was possible with both — but it made a wrong call roughly three times more likely per run and lengthened the average ticket by nearly three model calls, which on a quadratic cost curve is not a 65% increase in price. None of the twenty-two tools was individually bad. They were bad as a set, and no unit test on any one of them could have shown it.
What to Merge, Split and Hide
Merge steps that are always sequential, as start_return does. Split tools that need different permissions, even when the business treats them as one flow: a return and a refund almost always travel together, and they are still two tools, because issue_refund carries a $150 ceiling and an approval gate that start_return does not need. Permission boundaries beat convenience every time, because merging across one means the looser rule wins.
Then hide everything the agent should never reach, and be deliberate about it rather than trusting the tool list to be the boundary. Sundry's finance systems expose seller balance adjustments, payout holds and manual ledger entries through the same internal gateway the agent's tools use. None of them is in the surface, none of them is reachable through a parameter on a tool that is, and the credentials the dispatcher holds cannot call them at all. A tool that is absent from the list is a suggestion; a tool the agent's credentials cannot reach is a control (Chapter 12).
Naming a Surface, Not a Tool
Names are usually chosen one at a time, which is how a surface ends up with get_order, order_details and fetch_order_info in it, each defensible on its own. Read the whole list aloud as a set instead, with the descriptions covered: search orders, get order, track parcel, search policy, start return, offer replacement, issue refund, message seller, escalate to human. Nine verbs, nine objects, no two that could substitute for each other.
Where the reading stumbles, so does the model. Sundry's first surface had search_orders and find_order — one returned a list, one took an id — and the pair was responsible for most of the wrong first calls before the rename. The vocabulary test costs a minute and catches overlaps that no amount of description writing will fix afterwards, because a good description cannot rescue two names that mean the same thing.
Dynamic Tool Lists
The tool list does not have to be constant for the whole run. Narrowing it by task type is cheap and works: Sundry's triage pass sees four tools — search_orders, get_order, track_parcel, escalate_to_human — because a ticket that only needs a delivery date should not have a refund tool within reach. A ticket routed to returns sees seven, dropping track_parcel and message_seller, which it never needs.
Two benefits, one caveat. Accuracy rises because the model chooses from a smaller set, and cost falls because the schema block shrinks. The caveat is that a changing prefix breaks prompt caching, so the token saving is smaller than it looks and can be negative on short tickets — measure it rather than assuming. This is the same idea that makes the multi-agent split in Chapter 10 work, and it is worth trying in one process before it is worth trying with several.
The Sundry Surface, Justified
Four tools read: search_orders when there is no order id, get_order when there is, track_parcel for the carrier's view, and search_policy for the rule that settles the case. Four act: start_return for the label and the pickup, offer_replacement for a stock reservation, issue_refund for money, and message_seller for the one channel that leaves Sundry entirely. The ninth, escalate_to_human, exists because a surface with no exit forces the model to invent one, and the invention is always a confident answer.
Two tools were removed, and both deletions are more instructive than the additions. get_customer went when search_orders started returning the buyer's name and email in its summaries — it was selected on 4% of runs and never contributed to an answer. cancel_order was folded into start_return after the eval set showed the model choosing between them at chance on delivered orders, where only one of the two is legal. Deleting a tool feels like losing capability and is usually the cheapest accuracy improvement available.
API-shaped — expose what the backend already has, which for Sundry is nine endpoints on orders alone. The model must sequence them correctly on every ticket, at one model call per step, and each seam between them is somewhere a run can stop half-finished. It is fast to build because it is generated, and it is the version that scored 76%.
Task-shaped — expose what a good support agent does, so a return is one call and the sequencing lives in code that was already tested. The surface is smaller, cheaper per ticket, and much less prone to mis-selection. The cost is that every tool is a small piece of product design rather than a wrapper, and somebody has to decide what a return actually means.
Choose task-shaped unless the agent's job really is to be a general client for your API. The generated surface is a different, worse system that arrives sooner — not a shortcut to the same place.
- Auto-generating the surface from an OpenAPI document — twenty-two overlapping names, a schema tax on every turn, correct first calls down to 76%, and a model that guesses between neighbours (Chapter 4).
- Splitting an always-sequential operation into three tools — three model calls, three chances to stop halfway, and a label with no pickup that the business has no process for (Chapter 8).
- Adding a tool for a case seen once — it is paid for on every turn of every ticket forever, and it competes for selection on all of them.
- Giving read and write tools the same shape because the symmetry looks tidy — they need different validation, different ceilings and a different audit record (Chapter 12).
- Judging the surface by unit tests on each tool — every tool passes, and the failure mode is selection between them, which only a labelled set can measure (Chapter 9).
- Design tools from a transcript of what a good human agent actually does on a ticket, not from the API index.
- Keep the list under about a dozen, and measure selection accuracy every time it grows.
- Narrow the visible tool list by task where the work splits cleanly, and measure the caching cost before keeping it.
- Delete tools the eval set shows are never selected, or never selected correctly — starting with the ones that feel harmless.
Knowledge Check
Going from nine tools to twenty-two added no capability. What did it cost?
- Correct first calls fell to 76%, wrong calls tripled, and tickets grew by almost three model calls
- Mostly schema tokens, which are re-sent on every turn and dominate the cost of a longer list
- Reliability of the individual tools, since the generated wrappers had far less error handling
- Latency on every tool call, because a longer tool list slows the model's response down on each turn
Sundry's returns almost always end in a refund. Why are start_return and issue_refund still two tools?
- They sit on different sides of a permission boundary, and a merge would let the looser rule win
- The two steps are not always sequential, so merging them would create a state that cannot exist
- A merged tool would cost two model calls anyway, so the split is free in both tokens and latency
- Only one of the two can be retried safely, and merging them would make the pair unretryable
Why does a surface generated from an OpenAPI document usually fail?
- It exposes endpoints rather than tasks, so names overlap and the model has to invent the sequencing
- The generated schemas are malformed, so the model cannot express arguments the dispatcher accepts
- It covers only part of the API, so the agent hits gaps on the tickets that matter most
- The generated tools are slower, because each one wraps an HTTP call with extra validation
Every tool has unit tests and they all pass. What does that tell you about the surface?
- Nothing about selection between tools, which is where the surface's real failures live
- Very little, since unit tests cannot establish that any individual tool behaves correctly
- That argument validation is sound, which is the main risk a tool surface actually carries
- That the surface is small enough, because a bloated surface would fail its own tests first
What is the honest tradeoff of narrowing the tool list by task — four tools for triage, seven for returns?
- Better selection and a smaller schema block, paid for with a prefix that caches less well
- Better selection, at the cost of weaker authorization, since the dispatcher now trusts the list
- Better selection, at the cost of a second agent and the coordination overhead it brings
- Better selection, at the cost of tickets that need a hidden tool being answered incorrectly
You got correct