Topic 42

The Third Week

Concept

The booking-status page has been live for two weeks and Waymark is pleased with it. Then the operations desk asks for one small thing: cancelled bookings should look different from waitlisted ones — a different colour, and a line about the refund window. Tessa passes it on expecting an afternoon, because the whole page took an afternoon.

It takes four days. And when it comes back, two things that used to work do not: group bookings show the wrong guest name, and a reference typed in lower case now finds nothing.

Nothing dramatic happened. Nobody was careless. What happened is that a demo became a product, and products get changed. This page is the accounting — what the unread pile actually costs, and where the cost really comes from, which is not where most people put it.

What the afternoon bought, and what it invoiced
Week one: applause
Described, built, clicked, described again — and a page nobody could break in ten minutes of trying.
Week three: the invoice
One small change costs four days, brings back two bugs the clicking never visited, and Friday's version cannot be restored because it was never kept.

Change Is the Real Workload

Here is the fact that makes software different from the brochure Tessa wrote last spring. The brochure was finished when it was printed. Software is never finished; it is only in use, and being in use is what generates the work.

Most of what a piece of software costs over its life is spent after the first version exists. New needs arrive from the people using it. Odd cases surface that nobody imagined — the group booking, the lower-case reference. Something outside changes and the thing has to keep up. Each of those is a change, and each change is somebody opening code that already works and altering it without breaking the parts that are not supposed to move.

Now look at what the afternoon loop was optimized for. Every round asked one question: does it work now? That question was answered brilliantly, thirty times in an afternoon. Nobody at any point asked the other question — can this be changed safely later? — and the other question is the one products live inside.

The Unread Pile Compounds

Go back through the afternoon and count what accumulated. Round four generated some code. Round nine changed it. Round seventeen patched round nine's change, which was already patching something nobody had read. By four o'clock the page was a stack of edits on top of edits, none of which had ever been looked at by a human being.

That pile has a specific cost, and it arrives now. When the operations desk asks for the cancelled-bookings change, somebody has to answer a question first: where in here does the status get decided? The junior cannot answer it. They wrote none of it, in the sense that matters — they described it. They can say what the page does, because they clicked it. They cannot say what is in it.

And — Chapter 1 again, the same sentence for the last time in this chapter — neither can the tool. Notes left on disk can tell it how the project is laid out; what nobody wrote down is why round nine changed round four. Asked on Monday, it opens the same files a stranger would, reads them for the first time, and forms a guess about what the previous rounds intended. Often a good guess. Never a memory.

So the four days are mostly not spent making the change. They are spent re-deriving what the code already does, badly, by two parties who both have to read it fresh — and the tool's re-derivation happens inside a window that holds only part of the picture at a time.

This is the thing to take away and repeat in meetings: when behaviour surprises everyone, there is no one who knows the code. The knowledge was never in any head. It was never in the tool either. It exists only in the code itself, which is precisely the document nobody has read.

The Two Bugs Nobody Caught

The group-name bug and the lower-case bug were not created by the change. They were let through by it, and there were exactly two places they could have been stopped.

The first is an automatic check. There is none. The whole safety net for this page has always been a person clicking around it — which is a decent net for the parts the person visits, and no net at all for the parts they do not. Whoever tried the new cancelled-bookings colour did not happen to try a group booking, and did not happen to type a reference in lower case. Nobody would. The net checked one afternoon's worth of clicking, and only the clicking that occurred to the clicker.

The second is being able to go back. Tessa asks the obvious question — can we put it back the way it was on Friday while we work out what broke? — and there is no answer. The files on the machine are the only copy, and they are the changed ones. Friday's version was not kept anywhere. It was overwritten, round by round, by a tool doing exactly what it was asked.

Tessa finds that genuinely absurd, and she is entitled to. She has been keeping versions of a prompt since Chapter 5 — version four of the review template still sits beside versions one to three, because she wanted to be able to go back and because she wanted to know what changed. The software her booking site runs on has less version history than her template for sorting guest reviews.

The Diagnosis, Precisely

Here is where the accounting has to be exact, because the wrong conclusion is available and it is popular.

The AI generated fine code. It generated fine code in week one and fine code in week three. Read those files today and a competent developer would find ordinary work — not elegant, not disastrous, the sort of thing a capable person writes on a normal day.

What failed is a list of three absences:

Nobody reviewed it. No human read any change before it was kept, so no human ever formed a picture of the whole. Plausible-looking code went in unexamined, thirty times in an afternoon.

Nothing tested it. There were no automatic checks, so every change was safe only as far as somebody happened to click.

Nothing was versioned. No history was recorded, so "put it back" and "what changed on Monday?" have no answers, in a week when both questions matter.

Those three disciplines were not invented for AI. They were invented for humans, because humans have always produced code that looks fine and turns out to have a hole in it. The disciplines exist because writing code that appears correct is easy and writing code that is correct is not — for anyone. Take them away and the same failure follows, whoever did the typing.

Which is a structure the reader has already met. Chapter 3 spent five pages on it: fluent output, trusted because it read well, unverified. That was the Harbourview Annex — a hotel written up so warmly that nobody thought to check whether it existed. This is that page in a different medium. Code that runs is fluency. Clicking around it is admiring the prose. Neither of them is verification, and this book has never once said trust it because it reads well.

A kitchen makes the same point, and it makes it all the way to the end. The dish was delicious. Three weeks on, a guest wants it less salty, and there is no recipe: not because tasting was a bad way to cook, but because tasting was the only record anyone kept. The cook has forgotten. The kitchen re-derives dinner from scratch, twice, and one of the two attempts is worse than the original.

So the lesson is not "don't use these tools". Milo uses them all day and ships things his agency will happily change in month six. The lesson is which disciplines came off, and the next page puts them back on without giving up the afternoon.

Common Confusions
  • "The AI wrote buggy code." The bugs are ordinary ones, of the kind every project produces. What is unusual here is that nothing was in place to catch them: generated and hand-written code need the same nets, and only one of these had its nets removed.
  • "Four days for a small change proves the tool failed." The tool did in week one exactly what it was asked, quickly and well. The four days are the unread pile's invoice arriving, and it would have arrived the same way if a human had typed every line unreviewed.
  • "So don't use AI to build real software." The next page is the counter-example, built by Milo with the same tools and no third week. The variable that decided the outcome was which disciplines were kept, not which typist was used.
  • "A better prompt on Thursday would have prevented this." A better prompt makes a better first version. It does not read the code back to anyone, does not check the group booking next month, and does not record what Friday looked like.
Why It Matters
  • "The demo worked, and then the product did not" is a pattern Tessa will meet from the buying side repeatedly. Three questions now have teeth: who reviews the changes, what runs automatically, and where is the history.
  • This is the book's trust doctrine proved in a second domain. Fluent output, accepted because it looked right, fails the same way in code as in prose — which means the reader now owns the general law and not just the Chapter 3 example of it.

Knowledge Check

Why does this page say change is where software's cost actually sits?

  • Building the first working version is by far the most expensive stage
  • Software gets more expensive to run the longer it stays live
  • Software is never finished, and being in use generates the work
  • The tools become less capable as a project grows past the demo stage

Why could nobody say what the booking page actually did?

  • The junior who built it had left the agency before week three
  • The generated code was written in a style humans cannot easily follow
  • Some of the files from the original afternoon had been lost
  • Nobody read it, and the tool has no memory of having built it

Which three disciplines does this page name as the ones that were skipped?

  • Review, tests, and versioning
  • Planning, documenting, and training
  • Estimating, budgeting, and reporting
  • Designing, deploying, and monitoring

Where does this page place the blame for the two bugs?

  • On the model, which produced poor-quality code for this project
  • On the missing nets, without which ordinary bugs get through unnoticed
  • On the junior, who was careless during the original afternoon
  • On the request, which was too vague for anyone to build safely

You got correct