AI-Assisted Development
by Sergey Okinchuk
Agents That Open Pull RequestsWhat It Costs

Chapter Fourteen · Cost, Evidence and Your Own Skills

Cost, Evidence and Your Own Skills

The honest accounting. Four topics count what an agent costs on all three of its meters, read the published evidence on whether it helps and why the studies disagree, keep the skills that supervision depends on, and draw the line where the loop is the wrong tool.

4 topics

By now Cragmoor has used the agent for months, and the question the team avoided at the start can be asked with numbers: what is it actually worth? Not in a demo, and not in the way it feels on a good afternoon, but in tokens, hours of review, rework and the team's own skills.

The chapter opens on cost as mechanisms: tokens, plan limits and the review time nobody budgets, with every price left to Chapter 15. It then reads the published studies of whether the tools help, why their results range from a 26% gain to a 19% slowdown, and how a team measures it for itself. It turns to the cost that no meter shows, the skills you stop exercising, and ends on the areas and situations where the loop of this book is the wrong tool.

Cragmoor runs through all four topics: the bill for the refund change, a month of the team's own counts, Kai after his months of vibe coding, and the payment and waitlist code the team decides to keep out of the agent's hands.

Four costs of a change, from the one that is not billed to the one that is
Skills
what the team can no longer check once it stops practising; shows up only when the agent is wrong
Rework
what review missed, arriving weeks later with someone else's name on the incident
Review
hours of engineer time per change; the largest meter on most teams
Tokens and limits
the only meter that sends an invoice

Topics in This Chapter

Topic 01
What It Costs
Why the 30th turn costs several times the 3rd, what a plan limit does to a half-finished task, and the review hours that outweigh the token bill in Cragmoor's month.
Evidence
Topic 02
Does It Actually Help?
Field experiments with 4,867 developers, a trial where experts were 19% slower and felt 20% faster, a follow-up its own authors called unreliable, and how to count on your team.
Evidence
Topic 03
Keeping Your Skills
The supervision paradox, a 2026 study where how people used the assistant decided what they learned, the skills that fade first, and the practice that keeps them.
Evidence
Topic 04
When Not to Use It
Code nobody can review, code you must trace at three in the morning, changes smaller than their prompt, loops that cannot close, data that cannot leave, and learning.
Evidence