Errors, Limits, and Retries

Milo is showing Tessa the classifier running against a handful of test reviews, and partway through, the screen fills with red. A 429 — too many requests. Tessa braces for the fixing to start.

Milo does nothing. He looks at the message, says "fair enough", visibly counts to himself, and sends it again. It goes through. "The error told me to slow down," he says, "so the fix is slowness."

That is a stranger sentence than it sounds, and unpacking it is this page. A service that says no, says nothing, or fails is not an unusual event — it is Tuesday — and the difference between a system that survives it and one that falls over is almost entirely a matter of what happens in the next two seconds. The beginner's instinct is to try again immediately. It is the single move that makes everything worse.

Something comes back wrongor nothing comes back

→

Read what it waswrong ask · transient · unknown

→

Fix it, wait longer and retry, or check before re-sendinga wrong request never heals

→

Throughwith every attempt on the bill

Rate Limits Are Policy, Not Failure

Providers cap how fast an account may ask. Usually two caps at once: how many requests per minute, and how many tokens per minute — because a hundred tiny asks and one enormous one are different kinds of load. Those caps are a rate limit, and going over one produces the 429.

Notice what a 429 is not. Nothing failed. Nothing is broken, nothing is misconfigured, and the model was never involved. A healthy server made a deliberate decision to serve you a little later, so that it can keep serving everyone. The limits exist because capacity is shared, and an account that could ask infinitely fast would be an account that could take the service down for everybody else.

Which is why hammering the re-send button is not neutral. Every retry inside the limit window is another request over the cap, so the punishment renews itself for as long as you keep pressing. People have spent whole afternoons in that loop, getting angrier, being the cause.

The fix is to wait, then try again — and to wait longer each time it happens again. A second, then two, then four. That pattern has a name: backoff. It sounds like giving up and is the opposite; it is the only strategy that reliably gets through, because it lets the window clear instead of continuously refilling it. Milo's counted beat was backoff done by hand. In a program it is a few lines and it is not optional.

Anyone who has redialled a busy phone line knows this in their bones. Pressing redial the instant you hear the tone gets you the tone again. Waiting a moment gets you through. The line was never broken; it was occupied, and occupation is a thing that ends on its own if you stop competing with yourself for it.

Why a Retry Is Not Free

Now the sentence this whole chapter has been building toward, and the one to carry out of the book: a retry is not free. There are two reasons, and Tessa already has both.

The first is money. Every attempt that reaches the model is billed in full — tokens in and tokens out, at the ordinary rate, exactly as reported in the usage block she read two pages ago. There is no discount for "this is the same question I already asked". Three attempts at one classification cost three classifications. A program that retries carelessly does not fail loudly; it succeeds expensively, which is much harder to notice.

The second is worse, and it comes from the timeout. When no response arrives, you do not know whether the work happened. So a blind re-send is not a second attempt at one job — it may be a second execution of a job that already completed. You will be billed twice, and something may have been done twice.

For a question, doing it twice is wasted money and a shrug. For anything that acts in the world, twice is an incident. If the request sent a confirmation email to a guest, the guest now has two. If it issued a refund, Waymark has issued two. If it posted a review to the tracker, there are two rows where one belongs. The model does not know it is repeating itself, and nothing downstream will stop it.

So the rule splits by what you know, not by how the failure felt:

• The request definitely did not happen — a 429 or a 500, an answer that arrived and said no. Retry, after a wait. The only cost is the tokens.
• You do not know whether it happened — a timeout. Do not re-send blindly. Check first, if there is any way to check: did the email go, does the row exist, is there a record on the provider's side. If the work cannot be checked and repeating it would be harmful, that is a decision for a person, not for a loop.

Which Errors Deserve a Retry at All

The other half of the discipline is knowing which errors will never improve, and the first digit from the status-code page does most of the work.

A 4 code other than 429 means your request is wrong. The credential is invalid, the body is malformed, something is missing. Send the identical request a thousand times and you will collect a thousand identical errors, because nothing about the situation changes between attempts. These do not heal. They get fixed: a bad key is replaced (the previous page), a malformed body is corrected (the page before that).

The retryable errors are the transient ones — the ones caused by a moment rather than by the request. A 429, because the minute will end. A 500, because whatever failed inside the provider is usually momentary. A timeout, with the caution above attached.

That is the whole triage, and it fits in a sentence: fix the wrong ones, wait out the transient ones, and think hard about the unknown ones.

What This Looks Like in Tools

Tessa is not going to implement backoff. She is never going to write the loop, and this page has not been teaching her to.

What it has been teaching her is what to expect from software that other people wrote. Every product Waymark plugs in either handles this properly or does not, and the difference is invisible in a demo and extremely visible at nine on a Monday morning when everyone hits the thing at once. So a question joins the list she already asks vendors: how does this handle rate limits and timeouts? A product that pauses and resumes politely is one that was built by someone who had this page's problems. A product that shows the user a red error and stops was not.

And when Milo's classifier runs against all 2,300 reviews in the next chapter, Tessa will see it slow down, pause, and pick up again — and she will not think it is broken. She will think: that is the rate limit, and it is being handled. Which is, in the end, what this chapter was for.