Agentic AI from Scratch
by Sergey Okinchuk
Prompt and Model VersioningThe Frameworks, Honestly

Chapter Fourteen · What You Built, and What Comes Next

What You Built, and What Comes Next

Eighty-eight per cent of the queue handled, six cents a ticket in model spend, 6% reaching a person with a summary attached, and an incident log with three entries. Five topics that map what you built onto what the frameworks sell, give a straight build-or-buy answer with the arithmetic shown, report Sundry's whole arc including the two changes the team was proudest of that moved nothing, and separate the problems that are open from the ones that are merely hard.

5 topics

This chapter stops teaching and starts accounting. Every abstraction the agent ecosystem sells has now been built by hand somewhere in this book — the loop in Chapter 1, tools and schemas in Chapter 3, context assembly in Chapter 5, retrieval in Chapter 6, the state machine in Chapter 7, the eval suite in Chapter 9, the boundaries in Chapter 12, the trace and the cost model in Chapter 13. That is what makes a framework's documentation readable as a set of choices somebody made on your behalf, which is the only useful way to evaluate one.

Two of these pages are written to be thrown away, and they say so. Topic 76 is the second and last quarantined page in the book: product names and feature sets as of 2026, isolated so the other thirteen chapters do not go stale when a library ships. Topic 79 dates every claim it makes for the same reason, and separates the problems that are open from the ones that are hard engineering with known answers — a distinction that decides whether a design gets an estimate or a human handoff, and getting it wrong is how a team loses a quarter.

In between sits the accounting. Topic 78 reports Sundry's full arc with the changes that produced each move, the three incidents and what each one cost, the composition of the 6% that still reaches a person, and the two improvements the team was proudest of that turned out to do nothing. There is no summary chapter after this one. The last topic hands the reader to the courses that teach the ordinary half of running a service, and to the three things worth doing in the first month — then names the assumptions the next model release will not rescue.

The whole arc in one line — four stops between the endpoints, and not one of them is a better model
$0.41 · 61%the forty-line agent, and nothing reaching a person on purpose
→
CachingChapter 5 · a layout change
→
RetrievalChapter 6 · better material, not harder thinking
→
EvalsChapters 8 and 9 · the instrument, not the improvement
→
ControlsChapters 12 and 13 · ordinary code around the model
→
TODAY$0.06 · 88%6% reaching a person, with a summary attached

Topics in This Chapter

Topic 76
The Frameworks, Honestly
Eight framework words mapped back to the chapters that built them, what libraries genuinely give you — a tested loop, streaming, persistence, a real graph abstraction — and the three things they hide that you need on the day something breaks. A selection checklist that starts with "can I see the exact request", written as of 2026 and expected to age fastest.
Frameworks
Topic 77
Build or Buy
A hosted agent product priced against an owned one with engineering time in the sheet, including the crossover volume — somewhere between 800 and 2,600 tickets a week on Sundry's numbers, and Sundry runs at 4,200. The four things that never transfer either way: the tool surface, the policy library, the eval set and the approval design.
Procurement
Topic 78
Sundry in Production
Resolution 61% to 88%, cost $0.41 to $0.06, escalation 6%, and every move attributed to the change that produced it — including the eval set, which moved no metric and redirected everything after it. The three incidents with what each cost and how it surfaced, the two changes that moved nothing, and the three things Vera would do differently.
Results
Topic 79
What Is Still Unsolved
The reliability ceiling that compounding per-step accuracy imposes, grading work whose criteria you cannot write down in advance, prompt injection with no general defence as of 2026, long-horizon memory, and depth as a tradeoff rather than a problem. Every claim dated, with what would change it and where you would see that.
Open Problems
Topic 80
Where to Go Next
The ordinary half of running an agent belongs to the observability, security, container and Linux courses in this catalogue, and the model half to Machine Learning from Zero — with the honest note that fine-tuning shapes behaviour and adds no knowledge. Then the first month: an eval set for something you already run, one traced production run, and the threat model for the tools already exposed.
Next Steps