Measuring Regression

None of the last three pages fits Plateful's other flagship model. The delivery-time predictor from Chapter 3 doesn't say yes or no — it says "38 minutes", and dinner arrives in 45. Was that a false positive? The question has no meaning; there is no cell for "seven minutes late". Regression models — the ones that predict a number — need their own honesty toolkit, and it is built on a different question: how far off, how often? The typical size of the miss, and whether rare huge misses matter more than everyday small ones.

Think of judging a darts player. The natural score is the average distance from the bullseye across all their throws — a steady summary of typical skill. But now imagine a game where one wild throw loses everything: suddenly you don't care about the average, you care about the worst throws. Both are fair ways to judge the same player; they reward different players. Regression measurement has exactly these two instincts, and this page names them — then hands the darts back.

The Average Miss

Start with one prediction. The model said 38 minutes; the courier took 42. The miss is 4 minutes — and we keep only its size. Off by 4 early or 4 late, the miss is 4; direction gets dropped, for a reason the Common Confusions box makes vivid. Do this for every prediction in the held-out test set from Topic 13, then average the sizes. Say five deliveries missed by 4, 9, 1, 7, and 4 minutes: add them up — 25 minutes of total miss — divide by five deliveries, and the answer is 5. This number has a name: the mean absolute error, or MAE — "mean" for average, "absolute" for keeping only sizes.

Its translation into English is the most honest single sentence you can say about a regression model: "we're typically off by about 5 minutes." No jargon survives the translation, and nothing important is lost in it. When the data team reports MAE, that sentence is what the number means — and it is the sentence Iris carries out of the room to everyone else.

When Big Misses Matter More

But an average can hide two very different worlds. Compare two delivery-time models that both score a typical miss of 5 minutes. One is steadily wobbly: nearly every prediction lands within a handful of minutes of the truth. The other is calm with catastrophes: nine deliveries in ten are off by a mere 2 minutes, and the tenth is off by more than half an hour — nine small misses and one huge one average out to the same 5. Same MAE. Not remotely the same product: nobody cancels over a 5-minute wobble, and everybody remembers the pizza that came 32 minutes after the app promised.

When the disasters are the enemy, teams reach for a measure that punishes big misses far more than small ones. Here it is in one sentence, and one sentence is all this course needs: RMSE (root mean squared error) squares each miss before averaging, so one 30-minute miss outweighs ten 3-minute ones — a doorknob for a future, more mathematical deep dive, not a tool you need to operate here. What you keep is the instinct: ask the data team not just "what's the typical miss?" but "is that steady wobble, or calm plus catastrophes?" — because the average alone will not tell you.

Off by Five Minutes — of What?

The same 5-minute miss also reads differently depending on what was promised. On a 15-minute pizza run, 5 minutes is a third of the whole wait — customers feel it, and the promise looks broken. On a 90-minute holiday-banquet order, 5 minutes is a rounding error nobody will ever notice. The absolute miss is identical; the experience is not.

That is the difference between absolute error — the miss in real units, minutes or dollars — and relative error — the miss as a share of the true value, roughly "off by a third" versus "off by one part in eighteen". Neither frame is more correct; they answer different questions. Reporting to the kitchen-operations team, minutes matter. Judging how broken the promise feels to a customer, the share matters. Pick the frame that matches the experience you are managing — and when someone quotes an error, ask which frame it is in before reacting.

The Dumb Rival Returns

Finally, the floor from Topic 15 — the one that just unmasked accuracy in this chapter — shows up here too, now in a regression problem. The zero-intelligence strategy here is: always predict the historical average. If Plateful deliveries have averaged around 40 minutes forever, predict 40 for every single order — rush hour, rainstorm, Tuesday lunch — and you have a "model" that took thirty seconds to build and is embarrassingly hard to beat.

Suppose that always-40 guesser is typically off by about 9 minutes. Now the real model's "typically off by 5" finally means something: it earns back 4 of those 9 minutes — that is its actual contribution, the part the fancy machinery added beyond a one-line rule. And if a model is typically off by 8 and a half against that same floor? It is barely earning its keep, whatever its dashboard looks like. This comparison — model versus dumbest rival — is about to become the discipline of the next page, for every kind of model at once.