When the Product Breaks

It is a Tuesday, and three things go wrong before eleven. The CRM's suggest-a-reply button spins and never returns anything. The airline's support bot answers a straightforward question about baggage with something oddly beside the point. And Tessa's own review script, running a late batch, stops with a message on the screen.

A year ago there was one sentence for all three of those: the AI is down. It is a comfortable sentence, and it is almost always wrong, and it leads nowhere — you cannot act on it, escalate it, or wait it out with any confidence.

Tessa is not going to say it again, because she now owns a map of everything standing between her and an answer. This page turns that map into a routine: what breaks, how to find out which, what she can fix herself and what she hands to somebody else with the work already done.

The Three Places It Breaks

Between Tessa pressing a button and a sentence appearing on her screen there are exactly three regions, and every failure she will ever meet lives in one of them.

The product. The vendor's own servers and the vendor's own code — the gathering, the building, the shaping from this chapter's first page. Their systems can be down. Their payload-building can have a bug. Their budget dials can be set somewhere unhelpful, and their account can be over its rate tier because ten thousand other customers are also having a Tuesday morning.

The wire. Everything between the two: Waymark's office network, the internet in general, and the timeouts that give up when nothing has come back. Rarer than it used to be, still the first thing to rule out because ruling it out takes five seconds.

The provider. The far end. A genuine outage, a degraded service that is slower than usual rather than dead — or the subtle one that gets its own section below, where the model itself changed under everybody's feet and nothing failed at all.

Three regions. The entire diagnostic value of this page is that Tessa can now say which one she is in, and she can do it without knowing anything about how any of them are built.

The Diagnosis Kit

Three moves, in order, none of them requiring an engineer.

Try the provider's own chat. This is the sharpest single tool she has, and it works because of what this whole chapter established: the product and the chat box reach the same kind of machine over the same kind of request. So open the ordinary chat box and ask it anything. If it answers instantly and well, the provider is fine, and the problem is the product's or the wire's. If it is also slow or refusing, the problem is upstream of everyone and there is nothing for Waymark to do but wait.

One honest note, so this is not oversold: a product may be built on a different provider from the chat box Tessa happens to use, in which case a healthy chat box proves less than it seems. It is still the fastest first move available, and when both are sick at once the answer is nearly always upstream.

Check the status pages. Providers and serious product vendors publish a page that says whether their service is healthy, degraded or down, updated during incidents by people whose job that is. Those pages exist for exactly this moment and almost nobody outside engineering knows to look at them. Find both, once, on a calm day, and put them in a folder in the browser. Ten seconds on a bad morning, and it frequently ends the investigation.

Read the shape of the error. If a number is visible anywhere — in the script's output, in a developer console, in a support reply — Chapter 7's rule still holds and does all the work. A 4 means the asking side, a 5 means the answering side, and 429 means somebody is over a rate limit.

That last one is worth a moment, because it changes what Tessa can say in a ticket. A 429 arriving from a vendor's feature is not her problem and never was: it means the product is over its rate tier with the provider. Either their traffic outgrew what they bought, or their handling of a busy minute is not what Chapter 7 would recommend. Either way, "your feature is returning rate-limit errors on Tuesday mornings" is a sentence that lands in a vendor's support queue very differently from "the AI is not working".

Waymark's office has a lamp that dies about once a year, and everyone in the office already knows this routine perfectly in another domain. Try another socket. Glance out at the neighbours' windows. Only then call somebody — and when you do, you say "it is the circuit, not the lamp, and the rest of the street has power". Nobody in that office is an electrician. Isolating was never the electrician's skill; it is the thing anyone can do, and it is most of what makes the call useful.

When Nothing Breaks and Everything Changes

Now the eerie failure, and it is the one nobody warns beginners about.

Nothing errors. Every request returns a clean 200, every status page is green, every button works. But the answers are different from last month. The summaries are longer and blander. The classifier that put a certain kind of review into "mixed" now puts it into "complaint". The tone of the support bot shifted slightly and nobody can quite say when.

Two things cause that, and neither of them is visible from outside.

The provider updated the model. Models are improved and replaced on the provider's schedule, and an update that is better on average can still be different on your particular task — which is the only thing you care about.

Or the vendor edited the system prompt. Somebody at the product company changed a line in the standing instructions on a Thursday afternoon, for perfectly good reasons, and the behaviour Waymark had built a habit around moved.

No status page announces either. There is no error to read, no code to look up, no support queue that has heard about it. Which leaves Tessa exactly one alarm, and she has had it since Chapter 5: her own checks, on her own output. The rule-checks that flag out-of-range categories. The dozen sampled rows read against their reviews. The flagged-row count that runs at 31 every season and comes back at 300 this time.

Those checks were built to catch the model being wrong. Their second job, which nobody designs them for, is to catch the model being different — and for a silent change they are the only alarm there is.

Teams that depend on this work handle it by asking their provider to keep them on a fixed version of the model rather than the newest one, and by re-testing deliberately when they do move. How that is arranged, and what it costs, is a production question that belongs to the course that builds these systems. What belongs here is knowing that drift is a normal maintenance fact rather than a betrayal, and that noticing it first is a thing Waymark can do for itself.

Fix, or Escalate Well

The three failures from Tuesday morning sort themselves cleanly, and that is the point of the page.

Her own script is hers to deal with, and the error on the screen says 500. Something failed inside the provider, momentarily, and those twenty lines were written to wait out a rate limit and nothing else — which is a perfectly reasonable place for a twenty-line program to stop. The remedy is Chapter 7's: wait a few minutes and run the reviews it had not reached yet. Because each review is a separate self-contained request, nothing is half-done and nothing is at risk of being done twice. And if it happens on every run, the fix is not heroics — it is asking Milo to treat a 500 the way the program already treats a 429.

The vendor's features are not hers to fix, and that is where the isolation pays. Compare two tickets.

One says: the AI in your product is not working, please advise. It will be answered in three days by somebody asking her to clear her cache.

The other says: since about 9:40 this morning your suggest-a-reply button spins and never returns. The provider's own chat box is answering normally, and our other AI tools are fine, so this looks like your side rather than an upstream outage. Your status page shows all green. Three of us on two different networks see it.

That second one contains a start time, an isolation result, a check of their own published status, and a scope. Anyone who has ever worked a support queue can tell you what happens to it: it goes to somebody who can actually look, immediately, because the cheap questions have all been answered in advance.

Which is the whole argument of this page compressed into one sentence. The person in the office who can localize an AI failure in five minutes is not doing an engineer's job — they are doing the part of it that does not require being an engineer, and it is the part that decides how fast anything gets fixed.

And it closes this half of the book honestly. Products built on the wire inherit the wire's failures; there is no version of this technology where a button always works. Knowing the machinery means knowing where it breaks, and that is the only kind of trust worth having in a tool.

The next chapter takes all of it — the request, the payload, the settings, the checks — and points it at the strangest and fastest-moving use of these models yet: the ones that write software. Milo has been living in that world for two years, and it has not been kind to everyone who wandered into it.