The Tool-Call Round Trip

One full circuit: the model asks, your dispatcher validates and executes, the result comes back as a message, and the model reads it on the next call. Mechanically it is five steps and about twenty lines of code, and most teams get the mechanism right on the first afternoon.

The engineering that matters is in what the result contains. A tool result does not disappear when the caller is done with it: it is context, re-sent and paid for on every remaining turn of the run. Shaping it at the boundary is the single biggest win available in an agent's first month, and it happens in your code rather than in a prompt.

The Circuit, Step by Step

Five steps, in order. The model's reply contains a request block carrying an id, a tool name and an arguments object. Your dispatcher validates the name and the arguments against the schema you published. It executes the function. It appends a result message to the array, tagged with the same id the request carried. Then the loop calls the model again with an array one message longer, and the model reads the result as context like any other message.

The id is the step people drop, and it costs nothing until the day the model asks for two tools at once. When it requests get_order and track_parcel in the same turn — which it should, since the two are independent — two results come back and the pairing is by id, not by arrival order. Get that wrong and the model reads a carrier scan as the contents of an order record. It will not raise anything; it will answer the buyer, confidently, from nonsense.

Shaping the Result

Sundry's orders service returns about 6 KB of JSON for one order: 68 fields, a status history with three timestamps per transition, per-line tax classes, warehouse pick locations, and the payment processor's own risk metadata. Chapter 2 measured that result at 900 tokens inside the transcript, the single largest recurring item in a Sundry context. The support agent reads fourteen of them.

# raw upstream payload: ~6 KB, 900 tokens in the transcript
{"order_id": "SU-88421", "created_at": ..., "updated_at": ...,
 "channel": "web", "locale": "en-GB", "warehouse_ref": "WH-EU-3391",
 "status_history": [ ... 14 entries, three timestamps each ... ],
 "line_items":    [ {"sku": ..., "supplier_sku": ..., "tax_class": ...,
                      "pick_location": "A-14-3", "weight_g": 18400, ... } ],
 "payment":      {"psp_ref": ..., "auth_code": ..., "risk_score": 0.03, ...},
 ... }

# what get_order actually returns: 340 tokens, fourteen of them
{"order_id": "SU-88421", "placed": "2026-03-08", "delivered": "2026-03-10",
 "seller":  {"id": "SLR-2210", "name": "Ashcombe Furniture",
             "type": "marketplace"},
 "items":   [{"item_id": "ITM-1", "name": "4-shelf unit, oak",
              "price_cents": 11800}],
 "charges": [{"kind": "delivery", "cents": 995},
             {"kind": "delivery", "cents": 995}],
 "refunded_cents": 0,
 "seller_description": "...120 words written by the seller, untrusted..."}

The shaped version keeps the order id, the two dates, the seller and whether they are a marketplace seller or Sundry's own, the items with prices in cents, the charges, how much has already been refunded, and the seller's product description. Everything else is gone: no warehouse references, no tax classes, no risk scores, no status history. It is 340 tokens instead of 900, which is roughly 600 tokens off every turn from that point on, and running the eval set both ways gives the same resolution rate. Nothing was lost because nothing that was dropped was ever read.

The discipline behind it is one sentence: a tool's job is not to return the API's response, it is to return the answer to the question the tool exists to answer. Two fields on that list earn their place for reasons the raw payload does not make obvious. refunded_cents is there so the model can see that money has already moved on this order — the check that Topic 18 turns into a rule. The two identical delivery charges are there because the duplicate is half the buyer's complaint, and a summarized total would have hidden it.

Results Are Context, Not Return Values

An ordinary function's return value is consumed and forgotten. A tool result is appended to the transcript and re-sent on every subsequent call, because the model retains nothing between them. On a ticket that runs to Sundry's twelve-turn limit, a 900-token result that arrives on turn 3 is sent again nine times — 8,100 input tokens for one order lookup. The shaped 340-token version costs 3,060 for the same information. That difference is invisible in any single request and obvious on a monthly bill.

Cost is only the half you can measure easily. The other half is attention: everything in the context competes with everything else, and 900 tokens of warehouse metadata sits between the system message's constraints and the model's next decision. This is the mechanism behind the drift Chapter 5 opens with, where an instruction stated on turn 1 stops governing behaviour once fifteen turns of material have stacked up underneath it. Trimming results is a quality change wearing a cost change's clothes.

Prose or JSON

The deciding question is what the model has to do with the output. For a structured lookup it will pull specific fields, compare them and cite them, and a compact JSON object with plain key names is the cleanest possible form. For a policy passage it will read, weigh and quote, and JSON actively hurts — a paragraph chopped into an array of sentence strings reads worse than the paragraph did.

Keep each tool's format stable. Switching one tool between prose and JSON depending on how much came back forces the model to re-derive the shape mid-run, and it costs accuracy on exactly the tickets where more data came back — which are the complicated ones. Retrieval results get their own treatment in Chapter 6, where the format question is bound up with chunking and ranking.

Latency Inside the Loop

A tool call sits between two model calls, and the customer waits for all three. track_parcel reaches a carrier API that takes about 3 seconds on a good day, which lands directly on Sundry's target of 9 seconds to first useful message. Nothing in the model's own latency budget can absorb that, and no prompt change makes an upstream faster.

The main lever is parallelism, and it is available more often than teams use it. When the model asks for three independent lookups in one turn, running them in sequence costs the sum of their latencies — three 2-second calls become 6 seconds of dead air — and running them concurrently costs the slowest one. The two requirements are the ones already stated: preserve each request's id so results attach correctly, and append the results in a stable order so two identical runs produce identical transcripts. The rest of the latency work, including streaming the first sentence to the customer while tools are still running, belongs to Chapter 13.

Truncation Rules

Any result that can be arbitrarily large needs a cap at the tool boundary, chosen by you rather than discovered by the context window. search_orders on a wholesale buyer with 340 orders will happily return all of them. The cap belongs in the tool: return the ten most recent, and say so — showing 10 of 340 orders, most recent first; narrow with a date range. That sentence is the difference between a model that asks a good follow-up question and one that tells the buyer they have ten orders.

Two rules make truncation safe. Never cut mid-record: a JSON object sliced at a byte offset is read as complete, and the model answers on a fragment without noticing. And always provide the next step, whether that is a page token, a date range, or an explicit instruction to narrow the query. A cap with an honest marker and a way forward costs three lines in the tool and removes a class of confident wrong answers that is almost impossible to spot in a transcript.