Working with LLMs from Zero
by Sergey Okinchuk

Chapter Eight · Inside the Product

The Model Inside a Product

Every AI feature Tessa uses is a program filling in the request she learned to read: gather, build, send, shape. This chapter names the standing instructions that give a product its personality, the two settings that decide how it answers, and then reads the one program in this book — twenty lines that classify 2,300 reviews and turn a $19 month into $6. It ends on the Tuesday when everything stops working.

The last chapter took the costume off one chat box. This one discovers that every AI feature Tessa has ever used was wearing the same costume — the support bot, the suggest-a-reply button, the summarize button that appeared in the booking system last month. Underneath each of them is a program gathering some material, filling in the request she can now read, and dressing the answer up for the screen.

Five pages take that apart and then build one. What an AI feature is made of and where products actually differ from one another. The standing instructions a product slips into every request, which explain almost everything odd a bot has ever done — and where a product's memory of you really lives. The two settings that decide whether a feature is crisp or surprising, and why an answer sometimes stops mid-sentence. Then the centrepiece: the twenty-line program that classifies Waymark's 2,300 season reviews, read out loud line group by line group, with every line traceable to a chapter that taught it — and the $19 first month turned into $6. And last, what to do on the Tuesday when it all stops working, which turns out to be something a non-engineer can do rather well.

Four features, one request
Different fronts, identical machinery
What Tessa pressesan airline's support botsuggest a reply, in the CRMsummarize this threadthe booking system's review dashboard
What each one sendsstanding instructionswhatever material it gatheredthen the words you typedtwo settings, and the vendor's key
What comes backone message, and a receiptshaped into a bubble, a pane, three bulletsbilled to the vendor's account
Products cannot differ in the machinery, so they differ in what they gather and what they instruct. Feature quality is mostly payload quality.

Topics in This Chapter

Topic 35
Every AI Feature Is a Request
Gather, build, send, shape: the four steps behind every AI feature ever shipped. Where products genuinely differ, whose key pays for the tokens, and the four questions that turn a vendor demo into an evaluation.
Concept
Topic 36
The System Prompt
The standing instructions a product puts at the top of every request: identity, tone, its own rules, and the facts it injects — including your name. Why they steer rather than bind, and how to read product behaviour backwards.
Concept
Topic 37
Settings That Change the Answer
The two dials riding along in every payload. The randomness dial, named once as temperature, and the length cap that stops an answer mid-sentence. Where each gets set, and why the rest can stay in the box.
Concept
Topic 38
Tessa's Twenty Lines
The one program in this book, read in words: setup, request, send-and-wait, check, write. Every line comes from an earlier chapter, 2,300 reviews go through it in forty minutes, and the $19 month becomes $6.
Concept
Topic 39
When the Product Breaks
Three places a feature can fail and two checks that tell you which. Status pages, error shapes, the silent drift that no status page reports, and the escalation that gets answered on the same morning it is sent.
Concept