Tool Schemas the Model Can Use

A schema is documentation written for a reader who cannot ask a follow-up question. The name, the description, the parameter names and their descriptions are the entire brief the model gets, delivered once per turn, with no opportunity to clarify anything. If two tools read as though they do roughly the same job, the model picks between them at close to chance, and the result presents as a system that is generally unreliable for reasons nothing in your logs explains.

That makes schema writing an engineering task with a metric attached rather than a matter of taste. Sundry's search_orders and get_order overlapped in exactly that way. Two lines of description — no code change, no model change, no new tool — moved correct first-tool selection on a labelled set of 200 tickets from 71% to 93%. That is 44 tickets a run that stopped going the wrong way because somebody wrote a better sentence.

The Name Is Half the Battle

Names are read before anything else and carry more weight per character than any other part of the schema. A good one is a verb and an object: search_orders, get_order, issue_refund, track_parcel. A bad one is a noun with no action (orders), a verb with no object (lookup, execute), or a name that came out of the internal API's routing table (orders_v2_query). The model has to infer intent from that string on a turn where it has already read three thousand tokens of ticket and history.

There is a cheap test. Print the tool names in a list, cover the descriptions, and ask which one you would reach for given "where is my parcel" or "I never got the second box". If you hesitate between two names, the model will too, and its hesitation resolves as a coin flip rather than as a question. Renaming is a string in the declaration, a key in the dispatch table and a redeploy — do it early, because a name that has been in production for six months acquires log queries, dashboards and one runbook that reference it.

Descriptions Answer When, Not What

A description that restates the name teaches nothing. "Searches orders." tells the model exactly what it already inferred from the eight characters above it, and spends tokens doing so. What the model cannot infer is the selection criterion: which of two overlapping tools applies to the ticket in front of it right now, and what to do when the obvious input is missing.

# before: the description Vera shipped in the forty-line agent
{"name": "search_orders",
 "description": "Find orders by email, order id or date range."}

# after: two lines that state when to reach for it, and what it is not
{"name": "search_orders",
 "description": "Find a buyer's orders when you do NOT have an order id: "
                "takes an email address or a date range and returns summaries "
                "only (id, date, status, total). For the full detail of one "
                "known order, call get_order instead."}

# get_order was not touched. Selection accuracy: 71% -> 93% on 200 tickets.

Three things changed in those two lines, and each of them is a rule you can reuse. It states the condition under which this tool is the right one — no order id in hand. It states what comes back, so the model knows a summary will not answer a question about charges. And it names its neighbour explicitly, which converts a choice between two similar options into a decision with a stated tiebreak. Nothing about the implementation moved; search_orders did exactly the same thing before and after.

The failures it fixed did not look like selection failures. In production they looked like an agent that sometimes could not find an order: it would call get_order with an id it had guessed from the ticket text, get nothing back, apologize, and ask the buyer for their order number — on tickets where the email address alone would have found it. Read as transcripts, those runs look like bad luck. Measured against a labelled set, they are a wording bug with a number on it.

Parameters Carry Their Own Documentation

Every parameter gets a type, and the ones that need it get a format, a unit, an enum or an example. issue_refund takes amount_cents rather than amount, and the description says so again: integer, minor units, 11800 means $118.00. That redundancy is deliberate, because the failure it prevents is a hundred-fold error in both directions. A $118 refund submitted as 118 pays out $1.18 and generates a second angry ticket; the same number read as dollars where cents were expected pays $11,800 out of a seller's balance, and no eval that reads transcripts will catch either, because the sentence to the customer is correct in both cases.

Enums matter for the same reason and are underused. start_return takes a reason, and as free text that field becomes forty spellings of the same thing — "damaged", "arrived broken", "cracked side panel", "DAMAGED_IN_TRANSIT" — after which every downstream branch that matched on a string quietly stops matching. Six enum values (damaged_in_transit, wrong_item, not_as_described, faulty, arrived_late, changed_mind) make it a closed choice the model gets right nearly always, and the residual — the cases where it picks badly — tells you something real about whether your six categories match the queue.

Stating What the Tool Does Not Do

Boundaries prevent misuse more cheaply than error handling recovers from it. track_parcel reaches a carrier API that only holds scans for parcels dispatched in the last 90 days. Without that sentence in the description, a ticket about a delivery from last year produces three tracking attempts, three empty results, twelve seconds of latency, and a reply telling the buyer the carrier appears to have lost their parcel. With it, the model skips the tool and asks a useful question instead.

Write the negative space explicitly: the range the tool covers, the case it does not handle, and where to go instead. search_policy returns passages and never makes the decision. offer_replacement reserves stock and does not ship it. issue_refund moves money and cannot reverse a charge older than the payment provider's window. Each of those lines is under fifteen words, and each removes an entire class of wasted turn — which is not a substitute for a good error message when the model tries anyway, only a way to need it less often (Topic 16).

The Cost of Every Word

Schemas are re-sent on every turn, because the model retains nothing between calls. At Sundry the system message and the nine schemas together come to 1,100 tokens, and a ticket that runs to the twelve-turn limit therefore pays for that block twelve times: 13,200 input tokens before the buyer's own sentence is counted. Double every description in the name of thoroughness and that becomes roughly 21,000 on the same ticket, on 4,200 tickets a week.

Be clear about which half of that bill matters. The fixed prefix caches well — Chapter 5 shows the discount, and it is close to an order of magnitude — so the money is real but small. The cost that does not cache is attention: a nine-tool surface with a paragraph on each is 1,400 tokens of instruction competing with the ticket, the policy passages and everything the tools have returned so far, and instruction adherence degrades as that competition grows. Keep each description under three lines, spend the words on the selection criterion, and delete every sentence that restates behaviour the name already carries.

Measuring Schema Quality

The measurement is cheaper than it sounds. Take 200 tickets from the queue, label each with the tool a good human agent would reach for first, run one model call per ticket, and score exact match on the requested tool name. It costs one call per case instead of a whole run, finishes in a couple of minutes, and it is the only feedback loop that can tell you whether a wording change helped. Full resolution on the 120-ticket eval set is the number that matters in the end, but it moves for a dozen reasons at once and cannot attribute a change to a sentence.

Treat schema edits exactly like prompt changes: versioned in the repository, reviewed in the diff, and run through the set before and after. Vera's jump from 71% to 93% came with three rewrites in between that scored worse and were thrown away — including one that added a helpful-sounding paragraph about marketplace sellers and dropped selection by four points. That part rarely gets reported, and it is the whole reason the set exists: wording changes feel like improvements, and roughly half of them are not.