Why Tests Matter More Now
Before agents, writing the code was the expensive step, and the tests checked it afterwards. With an agent the code costs minutes, and the expensive step is knowing that it is right. The tests are the one artefact that says what "right" means in a form the agent can run, so they stop being a check on the work and become the specification of it.
Chapter 7 catalogued the ways agent code goes wrong, and most of them had one thing in common: nothing failed. The boundary flip, the rounding change and the second-zone deadline each went through the suite without a new failure. The pages that follow turn that around, so that the suite is what catches them.
The Ratio Changed
The refund diff was about 140 lines, produced in about two minutes. The tests that pin the 12-hour rule, the ones the next topic writes before asking for any code, are about 20 lines. Those 20 lines decide whether the 140 are acceptable. When producing code was slow, attention went to the code because that was where the effort was. Now the effort of producing it has collapsed, and attention belongs where the decision is.
The shift shows up in how review time is spent. Reading 140 lines of code tells you whether the code looks right. Reading 20 lines of tests tells you whether "right" was defined correctly in the first place, and no amount of care with the code makes up for a test that encodes the wrong rule. A reviewer who has ten minutes should spend most of them on the tests.
The Only Specification It Can Run
The task description from Chapter 5 and its examples are read and interpreted. A test is executed. The agent's loop repeats until its checks pass, so whatever check you hand it becomes its goal, literally. A test that states the rule makes the rule the goal. A test that states something weaker makes the weaker thing the goal, and the agent will meet it.
That is why the specification has to be a test and not only prose. "Exactly 12 hours ahead is refunded" in a task description is one sentence among many, and the agent may weigh it against the more common strict comparison it has seen everywhere. The same sentence as a failing test is a requirement the loop cannot end without meeting.
The examples Chapter 5 attached to the task are halfway there. "Cancelling exactly 12 hours ahead refunds $24.00" is precise, and the agent can read it. It still has to be read, weighed against everything else in the context, and remembered at the right moment. Turned into a test, the same example is checked on every run, by a program that does not weigh anything.
The Contract
The working agreement this chapter builds is short. You write or approve the tests. The agent changes the code until they pass. The agent does not change those tests without saying so, and you treat any change to them as Chapter 7 taught. The change merges when the tests you approved pass and you have read the diff.
The last clause matters. Tests replace "it seems to work", the judgement nobody can check. They do not replace reading. A diff that passes your tests can still contain the per-gym policy object nobody asked for, a new dependency or a changed log format, and only reading finds those.
The contract also changes what you say in a review. Ines's first question about any agent change is no longer "does this look right?" but "which test covers this?" A behaviour with a test is checked by the test, and the review can move on to whether it should exist at all. A behaviour without one gets a test before the review goes further, because nothing else will check it next time.
What Tests Cannot Say
Tests say whether behaviour matches the rules they encode. They say nothing about performance, security, readability or how the change fits the rest of the system. A green suite over the policy object says it does what it does; it says nothing about whether it should exist. Those questions stay with the review in Chapter 6.
Some of those gaps can be narrowed. A test can assert that a query runs in under a fixed number of database calls, or that a log line never contains a member's card number. But each of those is a rule someone had to think of and write down, and the agent will not think of it for you. What no test can say is whether the design is the one you would choose, and that question never moves out of the review.
The Module With No Tests
Where there are no tests, the agent has no contract and you have no evidence. At Cragmoor that is pricing, which has no tests at all. Asking an agent to change it is asking for code whose correctness nobody can check except by reading every line and guessing at the rules. The first job in such a module is pinning what the code does now, right or wrong, which Chapter 11 does before anything in it changes.
Cragmoor's Rule
After the refund change, Ines made it team policy. An agent's change may touch only behaviour that a test covers. Where the test is missing, it is written first, in its own commit, and reviewed as a specification before any code is requested. The rule costs a few minutes per task. The refund diff, which had no boundary test, cost a review round, a narrowed diff and a redirect.
The rule also changes where the work starts. Your next task, a change to the wording of the cancellation screen for pack bookings, began with you writing three tests and asking Ines to read them before any code was requested. Her review of the tests took ten minutes. Her review of the agent's code, against tests you had both already agreed, took fifteen, and found nothing the tests had not already settled.
- "The agent writes tests, so tests are cheap and matter less." It writes them fast. Whether they encode the rule is still your decision, and that decision is now the expensive step.
- "Tests are for catching regressions later." With an agent they are also the instruction now. A test it runs is a statement of the task that it cannot misread.
- "Careful code review makes tests unnecessary." You review one diff. Tests review every later diff, including the agent's next one that touches the same rule.
- "High coverage means the contract is complete." Coverage counts lines executed, not rules asserted. The refund module was 90% covered, a local figure, and nothing checked a cancellation at exactly 12 hours.
- "Tests slow the agent down." A two-second targeted run per attempt costs less than one review round spent finding what the test would have found.
- Review the test diff before the code diff on every agent change. The tests are the claim the code is judged against.
- Write or approve the tests that pin a rule before asking for the code that implements it.
- State in the project instructions file that existing tests are not edited without a stated reason.
- Add tests to a module before an agent changes it. No tests, no contract.
Knowledge Check
Why does the test become the specification once an agent writes the code?
- The agent cannot read task descriptions written in ordinary prose
- The agent's loop runs until its checks pass, so the check is its goal
- Tests are always correct, unlike the code an agent produces quickly
- The test runner is faster than the agent at reading and writing code
An agent's change passes every test you approved. What does that green suite still not tell you?
- Whether the rules your approved tests encode are met by the code
- Whether the tested paths run without crashing on the inputs used
- Whether the diff adds code nobody asked for, or fits the system
- Whether the cancellation at exactly 12 hours is now refunded
The refund module has 90% line coverage, yet the boundary flip passed the suite. What explains it?
- The flipped comparison sat on a line inside the uncovered ten percent
- Coverage counts lines that ran, and no test asserted the exact boundary
- Coverage tools do not measure lines that were written by an agent
- The agent edited the coverage report to show a higher percentage
You are asked to have an agent change pricing, which has no tests. What comes first?
- Ask the agent for the change and for tests of the new behaviour
- Read the module closely, then let the agent make the change
- Set a coverage target and let the agent write tests until it is met
- Pin what the module does now with tests, before any change is asked
You got correct