How Good Is Good Enough?
A different team, a different slide. Plateful's trust team has a model that reads a restaurant review and guesses whether it is genuine or planted, and the slide says it is right 93 times out of 100 — on held-out data, exam run properly, everything the last two pages demanded. The room looks pleased. Iris realizes she has no idea whether to celebrate. Ninety-three percent... compared to what?
That question is the whole page. A score, however honestly measured, means nothing on its own — it only means something against other numbers: what a dumb shortcut would score, what a human doing the job achieves, what the product actually needs. This page installs the comparison reflex. The measuring tools themselves — the honest ways to compute such scores — fill Chapter 6; here you learn where any score must be placed before it deserves an opinion.
A Number Needs a Floor
Start with the first comparison, and make it insultingly dumb on purpose. What would a strategy with no skill at all score on this job? For delivery times: always predict the average, thirty-five minutes for everyone. For reviews: always say "genuine," never flag anything. Such a strategy is called a baseline — the score you get for free, before any learning happens. It is the floor every model must beat to prove it has learned anything at all. This is a first look at the idea; Chapter 6 makes the baseline a standing tool of honest measurement.
Why the floor changes everything: suppose 94 out of 100 reviews on Plateful are genuine. Then "always genuine" — a strategy that couldn't catch a fake review if it were signed by the fraudster — scores 94%. Suddenly the model's 93% stops looking like a triumph; it is below the floor. A weather forecaster who brags about being right 85% of days sounds impressive — until you learn the forecasts are for a desert, where "always sunny" scores 90 without any skill. The desert is the floor. Every score you will ever be shown lives in some desert, and your first question is how sunny it is there.
A Number Needs a Bar
Beating the floor proves the model has skill. It does not prove the skill is enough — that depends on what the mistakes cost, and cost lives in the product, not in the score. A spam filter that is right 93% of the time delights everyone; a mislabeled email is a minor nuisance. Now put the same 93% into an app that answers "is this mushroom safe to eat?" Same number, and it is a lawsuit — seven poisonings per hundred foragers. The score did not change; the stakes did.
So above the floor there is a second line: the bar, the level the product genuinely requires. For fake reviews, the trust team has to weigh two different injuries — a fake review that slips through misleads customers; a genuine review wrongly flagged infuriates an honest restaurant owner. How the bar gets set from such costs is a Chapter 6 story with proper machinery behind it. What matters now is the shape: floor below, bar above, and the model's score meaning nothing except in relation to both.
Compared to the Human Alternative
There is one more comparison worth making, because in real products a model usually replaces or assists people who were doing the job already. That changes the arithmetic of "good." Plateful's trust team currently spot-checks reviews by hand — a few hundred a day out of tens of thousands, read by people who get tired, get bored, and go home at six. A model that is somewhat worse than a fresh human reviewer, but that reads every single review at three in the morning without its attention flagging, can still be a clear win. "Better than a tired human, cheaper than three shifts" is often the honest condition a model has to meet — not perfection, and not even matching a human at their best.
The comparison also runs the other way. If humans do the job at 99% and the errors are costly, a 93% model is not an upgrade, however honestly it was measured. Neither cheerleading nor cynicism survives this comparison — just arithmetic on the alternative.
Good Enough Is a Decision, Not a Discovery
Put the pieces together and something quietly important follows. Nothing in the data announces when a model is good enough. The floor can be computed, the score can be measured — but the bar is chosen, by people, weighing what each kind of error costs against what running the system buys. The trust team might decide 93% clears their bar once flagged reviews get a human second look; the mushroom app should never ship at 99. Same arithmetic, different decisions, both defensible — because "good enough" is a judgment about consequences, not a property of the model.
This is why "the model is good" is always shorthand. Spoken honestly, it means: "the model beats the dumb baseline by a sound margin, and it clears the bar we chose for this product, knowing what mistakes cost here." When the shorthand arrives without the long form behind it, ask for the long form. You will do a lot of that in Chapter 6, where the measuring tools live — and where the fraud-detection story shows how spectacularly a raw score can lie.
- "93% is objectively a good score." Meaningless until placed: against the dumb baseline (which might score 94 for free) and against the product's stakes (93 delights a spam filter and dooms a mushroom app). The number carries no verdict by itself.
- "The team should aim for 100%." Perfection is unreachable — labels carry mistakes and the world keeps changing — and chasing it invites the memorization trap from the last page. The goal is the chosen bar, honestly beaten.
- "It beats the baseline, so ship it." The baseline is only the floor — proof of skill, not of sufficiency. The real gate is the product bar, set from what each kind of error costs.
- You will never again hear "the model is 9X% accurate" without asking the two questions that give the number meaning: what does the dumb baseline score, and what does this product's bar require?
- This page frames Chapter 6, where the honest measuring tools live — and where you will watch a raw accuracy score lie outright about a fraud model.
Knowledge Check
A vendor announces their model is "right 93% of the time." What is the first question this page teaches you to ask?
- How many parameters does the model have?
- How long did the model take to train?
- What would the dumbest possible strategy score on this job?
- Was the score measured on the training data?
On Plateful, 94 of every 100 reviews are genuine. The fake-review model scores 93%. What does the "always genuine" baseline reveal?
- The zero-skill strategy scores 94%, so the model is below the floor
- Nothing — a baseline must be trained before it can be compared
- The model is within one point of the baseline, which counts as a pass
- That Plateful should ship the baseline instead of the model
A spam filter and a mushroom-safety app both score 93%. Why does the same number earn opposite verdicts?
- The mushroom app's score must have been measured without a proper test set
- Spam filters are tested on more data, which makes their scores more trustworthy
- One task is prediction and the other is not, so the scores cannot be compared
- The cost of a mistake differs — the product's bar, not the score, decides
Who or what ultimately determines that a model is "good enough" to ship?
- The test-set score — once it is high enough, the decision makes itself
- People — the team sets the bar by weighing what each kind of error costs
- The baseline — any model that beats the floor is ready for launch
- The model — its confidence numbers indicate when it considers itself ready
You got correct