Topic 76

The Frameworks, Honestly

Frameworks

Everything this book built by hand exists in library form, and most teams will use one. That is a defensible decision and a far better-informed one than it was fourteen chapters ago, because every abstraction a framework offers maps onto something you have already written. Its documentation now reads as a set of choices somebody made on your behalf, rather than a set of concepts to take on faith.

This is the second and last quarantined page in the book. Names, feature sets and capabilities below are stated as of 2026, and the page was written to be thrown away — the content audit rewrites it and Chapter 2's vendor page wholesale, and nothing else. What survives a rewrite is the map and the criteria.

The Map

Eight words cover almost every agent framework on the market, and each has a chapter of this book behind it. Read the table left to right when evaluating a library, so its documentation stops being a vocabulary lesson. Read it right to left when a framework surprises you in production and you need to know which chapter explains what it just did.

The framework's wordWhat it actually isBuilt by hand in
Agent, Runner, ExecutorThe four-step loop: send, read the stop reason, run the tool, appendChapter 1
Tool, Function, SkillA JSON schema sent to the model, plus a dispatcher your code ownsChapters 3 and 4
MemoryRe-sending history, compaction, or retrieval — three different productsChapters 5 and 6
Planner, Graph, State machineControl flow wrapped around the loop, with guarded transitionsChapter 7
Handoff, Crew, SwarmA boundary between agents and a typed envelope crossing itChapter 10
GuardrailA filter on input or output, with a false-positive rateChapter 12
Callback, Tracing, ObservabilitySpans over model calls and tool calls, carrying token countsChapter 13
Eval, ScorerA graded case set plus a judge with a measured agreement rateChapter 9

Two rows deserve suspicion before anything else. Memory is the loosest word in the field: it can mean a message buffer, a summarizer, a vector index, a store of extracted facts, or all four behind one constructor argument, and Chapters 5 and 6 spent twelve topics on why those have different costs and failure modes. Guardrail almost always names a filter, and Chapter 12 was blunt that a filter lowers a failure rate and is never the boundary. A team that reads it as a boundary ships the wound Chapter 12 opened.

What They Genuinely Give You

Start with the honest column, because it is longer than framework sceptics admit. A library hands you a loop thousands of people have run, so the edge cases you would otherwise meet in your third month are already handled: a malformed tool argument, an undocumented stop reason, two tool calls in one reply where your code assumed one. It hands you streaming, so a customer sees a first token while the run continues, retry and backoff around the model call itself, and tracing integrations that already speak to whatever platform your company collects spans in.

The graph or state-machine abstraction is worth naming separately, because it is real engineering rather than packaging. Chapter 7 built Sundry's five-state returns machine by hand: fixed phases, a guarded transition where money crosses, and a current state that survives a restart. A framework that gives you that as a first-class object — nodes, edges, persisted state, resume-from-here — has usually built it more carefully than you will on a Thursday afternoon between two incidents. If you borrow one thing from a library, borrow that one.

What a library genuinely hands you, against the three decisions it takes on your behalf — and each of those three is a chapter of this book
What you genuinely geta loop thousands of people have run
The edge cases you would otherwise meet in month three — a malformed tool argument, an undocumented stop reason, two tool calls where your code assumed one — plus streaming, retry around the model call, and tracing that already speaks to whatever collects your spans. The graph abstraction is the one to borrow: nodes, edges, persisted state, resume-from-here, built more carefully than you will manage on a Thursday between two incidents.
Hidden: context assembly and pruningbehind the word Memory · Chapters 5 and 6
What went into the buffer, in what order, and what was dropped when it filled — decided for you by a token limit and a summarizer whose prompt lives in the library's source. Sundry's drift turned on what was in the buffer and in what order, and finding it took a trace of the buffer as sent rather than a reading of the code.
Hidden: retry and idempotencybehind the word Tool · Chapter 3
The double refund happened because a retry decision was taken by a component with no way to know the write had already landed. The question for the documentation is not whether it retries — it is on what key, and whether it retries writes at all.
Hidden: the request itselfbehind one convenient line · Chapters 2 and 13
System instruction, message array in order, tool schemas, sampling settings, model version. Can you get those bytes, verbatim, for a run that already happened? If the answer is that there is a callback for it, find out whether it fires before or after the library's own transformation — the one that fires before tells you nothing about what the provider received.

What They Hide That Matters

Three things, and they are the three you need on the day something goes wrong. The first is context assembly and pruning policy: what went into the buffer, in what order, and what got dropped when it filled. Chapter 5 is an entire chapter on that one decision, and a framework makes it for you behind a token limit and a summarizer whose prompt lives in the library's source. Sundry's drift turned on what was in the buffer and in what order, and finding it took a trace of the buffer as sent rather than a reading of the code.

The second is retry and idempotency semantics. Chapter 3's double refund happened because a retry decision was taken by a component with no way to know the write had already landed, and the fix was a key derived from the intent. A library that retries failed tool calls on your behalf has made that decision for you, and the question for its documentation is not "does it retry" but "on what key, and does it retry writes at all". The third is the request itself.

One convenient line, and four policies you did not choose
result = agent.run(ticket)

# what went into messages, and in what order          -> Chapter 5
# what was dropped when the buffer filled              -> Chapter 5
# whether a failed tool call was retried, and on what key -> Chapter 3
# which model version and sampling settings served it  -> Chapter 13

In words: one call runs the whole agent, and four decisions this book spent chapters on are taken inside it by defaults nobody on your team has read. That is not an argument against frameworks — it is an argument for being able to see all four when you need them, which is where the selection criteria start.

Choosing on Criteria Rather Than Popularity

The first question is whether you can get the exact request, verbatim, for a run that already happened. Not a log line naming which tools were available: the bytes. System instruction, message array in order, tool schemas, sampling settings, model version. If the answer is "there is a callback for that", find out whether it fires before or after the library's own transformation, because the one that fires before tells you nothing about what the provider received. Every other debugging question in this book reduces to this one.

Four more, in the order they decide outcomes. Escape hatches — can you replace the context assembly or the retry policy without forking, and is that documented or a private attribute somebody found in the source. State persistence — is the run's state in your database in a schema you can query during an incident, or in the library's own store in a shape only its code reads. Observability — does it emit spans in a standard your company already collects. Licence and maintenance — who pays the maintainers, how many organizations hold commit rights, whether releases are pinnable and deprecations arrive with notice. A library with one corporate owner and one active maintainer is a single point of failure in your control flow, independent of how good the code is.

Capability is deliberately absent from that list. Every library named at the bottom of this page can run a support agent with nine tools, retrieval, a turn limit and an approval gate. The criteria above are about the third incident rather than the tutorial, which is why they are the part of this page most likely to still be true in two years.

The Middle Path Most Teams Land On

Own the loop and the tool dispatcher. Borrow tracing, eval tooling, prompt and version management, and run persistence. That is where most teams that have run an agent for a year end up, and the reason is structural. The loop and the dispatcher decide your product's behaviour and absorbed every incident in Chapters 8 and 12, so not understanding them is a debt that comes due at the worst possible moment. Tracing and eval infrastructure are undifferentiated, and rebuilding a scoring dashboard out of pride costs a fortnight and produces something worse.

Sundry ends this book in exactly that shape. The loop is around 300 lines including the dispatcher, the $150 ceiling and the intent record, in the same repository as the rest of the support service. Traces go to the platform Sundry's other services already report to, and the eval suite runs on a hosted tool with the 120-ticket set and the rubric versioned next to the prompt. Nothing about that is brave: it puts the code Vera gets paged about in a file she can open.

Reading This Page in Six Months

This topic and Chapter 2's vendor page are the two the content audit rewrites wholesale, and this one ages faster, because libraries ship weekly while the shape of a model API has been stable for years. Every capability claim above is stated as of 2026, and where a sentence disagrees with a library's current documentation, the documentation is right. If a page elsewhere in this book has gone stale, that is a defect in the writing rather than movement in the field.

Re-check it yourself rather than waiting for a rewrite. Answer this page's four questions against the library's current docs and source: can I print the exact request, can I replace the context assembly, where does run state live, and what does it do with a write that timed out. Then run your own eval set and compare resolution, cost and p95. That takes an afternoon and beats every comparison table on the internet, including the one above, because it measures the library against your queue.

Framework vs your own loop, for a team of three

Framework — a working agent in a day, tracing and persistence included, and a graph abstraction better than the one you would write this week. The costs arrive later: an upgrade treadmill on a fast-moving dependency, and debugging sessions that begin by reading the library's source to find out what it sent.

Own loop — about a week to the same place, and every byte inspectable from the first day. You write your own persistence and your own tracing integration, which is a fortnight of work with nothing a customer can see at the end of it, and you will get some edge cases wrong before production finds them for you.

The deciding factor is rarely capability, because both resolve tickets. It is whether the team can afford to be surprised by something it cannot see — which depends on what the tools can do, how quickly a mistake is noticed, and what undoing one costs. An agent that answers product questions can afford a library it does not fully understand. An agent moving money cannot.

Common Mistakes
  • Choosing by star count or by a benchmark blog post — neither measures observability or escape hatches, and those two decide what the third production incident costs.
  • Adopting a framework to skip understanding the loop — the first incident needs exactly the understanding that was skipped, and it arrives with a customer waiting.
  • Wrapping a framework in your own abstraction to stay portable — two layers of indirection forever, and the migration still costs a prompt rewrite and a cost re-measurement.
  • Rebuilding tracing and eval tooling out of pride — that is the half genuinely worth borrowing, and the fortnight it takes is one not spent on the policy library.
Best Practices
  • Evaluate against a checklist that starts with "can I get the exact request for a run that already finished", and treat a no as disqualifying rather than as a drawback.
  • Keep the loop and the tool dispatcher in your own repository, and borrow tracing, eval tooling, run persistence and prompt management from people who maintain them full time.
  • Pin framework versions exactly as you pin the model version, and run the eval suite against both versions before an upgrade reaches production (Chapter 13).
  • Re-read this page at every audit cycle and assume it aged fastest, because the names on it change more often than the machinery under them.
Comparable toolsLangGraph the loop as an explicit state graphCrewAI the loop plus multi-agent rolesPydantic AI typed tools and structured resultsMicrosoft Agent Framework, Mastra the same loop for .NET and TypeScriptLangSmith, Langfuse, Braintrust tracing and evals, the part worth borrowing

Knowledge Check

A framework's one-line agent.run() is convenient. Which decision does it take on your behalf that this book spent a whole chapter on?

  • Which provider serves the request, because the library selects the cheapest one at call time
  • What goes into the context buffer, and what gets dropped from it when the buffer fills up
  • Whether a tool is allowed to run, because the library's own dispatcher resolves permissions
  • How many turns a run may take, because libraries fix that internally with no setting shown

Two frameworks both run your support agent successfully in a prototype. Which difference should decide the choice?

  • Which one needed fewer lines of code to reach a working agent against the tools you run
  • Which one has more stars, more contributors and more integrations listed on its own docs site
  • Whether you can retrieve the exact request that was sent, for a run that has already finished
  • Which one scored higher on the published agent benchmark that both of the projects report

A framework ships a major version with a rewritten context manager. What does this book say to do before it reaches production?

  • Run the eval suite against both versions on your own tickets, comparing resolution, cost and p95
  • Read the changelog and the migration guide, and ship if no breaking API changes are listed there
  • Deploy to a canary and watch error rates, since a context change shows up as failures quickly
  • Pin the old version indefinitely, because a rewritten context manager is not worth the risk

Which split between building and borrowing does this book argue for, and on what grounds?

  • Own everything, since any dependency sitting in the model call path is an unacceptable risk
  • Borrow everything, treating the loop as undifferentiated plumbing a library maintains better
  • Own the loop and the dispatcher, borrow tracing, evals, persistence and prompt management
  • Own the tracing and eval tooling, borrow the loop, the dispatcher and the context assembly

You got correct