Working with LLMs from Zero
by Sergey Okinchuk

Chapter Seven · Under the Chat Box

The Request Under the Chat Box

Every message Tessa has ever sent left her browser as one small structured request and came back as one structured response. This chapter reads them. What pressing Enter really does, how machines ask and answer over the internet, the message itself line by line, the key that spends the money, what to do when things fail, and a $19 invoice explained down to the afternoon. The chat box has been a costume for six chapters. It comes off here.

Six chapters have treated the chat box as a given: a place where text goes in and text comes back. This chapter opens it. A developer from Waymark's contractor agency turns his laptop around, and what Tessa sees is not a mystery at all — it is one small, readable message travelling to a server, and one travelling back.

The six pages take that message apart. What pressing Enter actually does, and where a conversation's "memory" has really been living all along. How machines ask and answer over the internet, and what a three-digit status code tells you about whose problem you have. The message itself, read line by line — in the labelled-data format Chapter 5 already taught. The key that identifies the account and spends its money. What to do when the provider says no, says nothing, or fails, and why a retry is never free. And the bill: $19 for Waymark's first month, itemized down to the afternoon. By the last page the costume is off, and everything the earlier chapters could only describe has a mechanism underneath it.

The same moment, seen twice
One press of Enter
What Tessa has always seena box, a cursor, one sentencea reply that types itselfa conversation that seems to remember
What Milo turns aroundone request, crossing the internetthe whole transcript, sent againthe string that says whose account
What comes backa three-digit verdictthe reply, as one more messagea receipt: tokens in, tokens out
The chat box is a program that writes these requests. The window, the forgetting, the reset and the bill all stop being mysteries the moment it is visible.

Topics in This Chapter

Topic 29
The Chat Box Is a Wrapper
Milo turns his laptop around and the book's central mystery becomes mechanical: one press of Enter, one request. Where the conversation's memory actually lived, why the window is a limit on re-sent history, and who else can send these.
Concept
Topic 30
HTTP: Ask and Receive
The ask-and-receive rules behind every web page, taught as a working idea. Request and response, and the one rule that turns a wall of error numbers into a diagnosis: the first digit tells you whose problem it is.
Concept
Topic 31
The JSON Payload, Live
One real request and one real response, annotated line by line: which model, the message list with its three roles, the settings riding along, the reply and its token receipt. Then a pair to read cold, unaided.
Concept
Topic 32
The API Key Is a Password
The long string that says whose account is asking and whom to bill. Why a leak is spending rather than snooping, the four handling rules, why revoking comes before investigating, and what one key per purpose buys you.
Concept
Topic 33
Errors, Limits, and Retries
Milo hits a rate limit and does nothing, on purpose. Backoff, the two costs of a retry, why a timeout leaves you not knowing rather than knowing you failed, and which errors will never heal however often you send them.
Concept
Topic 34
The Bill: Paying per Token
Input and output priced separately, and Waymark's first $19 reconciled into three lines that explain themselves. The three dials that decide next month, and why a metered bill is cost made visible rather than cost made higher.
Concept