Precision and Recall
The last page ended with a question accuracy cannot answer: how does the fraud model do on the fraud itself? To answer it properly, the data team splits one vague question — "is the model right?" — into two sharp ones. When it points at a transaction, is it correct? And: of all the fraud out there, how much does it catch? Those two questions have names — precision and recall — and they are the working vocabulary of every real conversation about a classification model.
Both questions live on the same picture. Think of a mushroom picker walking out of the forest. Precision asks about the basket: of everything the picker gathered, what share is actually edible? Recall asks about the forest: of all the edible mushrooms that were out there, what share made it into the basket? A timid picker brings home a tiny, flawless basket and leaves the forest full. A greedy one sweeps up everything — including the poison. Hold that basket-and-forest split in mind; the fraud model faces exactly the same two-sided question.
Precision: When It Points, Can You Trust It?
Precision looks only at the model's alarms. Take yesterday's numbers from Topic 25: the detector flagged 16 transactions as suspicious, and 8 of those really were fraud. Precision is that share — 8 out of 16, or one half. In words: when this model points a finger, it is right half the time.
Low precision is the crying-wolf failure, and its cost lands on real people. Every wrong flag is an innocent customer whose dinner payment got blocked, a support call, an apology. Worse, it erodes trust inside the company: when half the alarms are false, the fraud team starts glancing past them — the same alarm fatigue that makes people unplug a twitchy smoke detector. A model nobody believes is a model nobody acts on, whatever its other virtues.
Recall: How Much of the Real Thing Gets Caught?
Recall ignores the alarms and looks at the fraud itself. There were 10 real frauds in the thousand transactions; the detector caught 8 of them. Recall is that share — 8 out of 10. In words: of all the fraud that actually happened, this model catches eight in ten.
Low recall is the opposite failure, and it is dangerously quiet. The two frauds that slipped through triggered no alarm, upset no customer, generated no support ticket — just money leaving politely through the front door, invoiced to Plateful weeks later as chargebacks. Crying wolf is loud and annoying; missing wolves is silent and expensive. That asymmetry is exactly why one number was never going to be enough.
The Tug-of-War
Here is the uncomfortable part: the two scores pull against each other. Remember from Topic 17 that a classifier outputs a probability, and a human-chosen threshold turns it into a verdict. That threshold is the rope in this tug-of-war. Lower it — flag anything even mildly suspicious — and the model catches more real fraud, so recall rises; but the flood of flags is mostly innocent, so precision falls. Raise it — flag only the near-certain cases — and almost every alarm is real, but more fraud walks past unflagged.
Concretely: set the threshold timid and the detector flags only 5 transactions, 4 of them real — precision four in five, but recall has collapsed to four in ten. Set it aggressive and it flags 45, catching 9 of the 10 frauds — recall nine in ten, but precision drops to one in five, and 36 innocent customers get their payments questioned. You do not escape this trade with a cleverer dial. You choose a point on it — and genuinely improving both at once takes a better model or better data, not a bolder threshold.
Whose Call Is the Balance?
So which end of the rope should win? That depends entirely on what each mistake costs — which makes it a product question, not a technical one. A spam filter leans precision-first: a spam email slipping into the inbox is a shrug, but a real email eaten by the filter — a job offer, an invoice — is a disaster, so the filter flags only when nearly certain. Disease screening leans the other way, recall-first: a false alarm means an unpleasant week and a follow-up test, but a missed case can mean a life; screening programs accept many false alarms to miss as little as possible.
Notice who decides. The data team can move the threshold anywhere in an afternoon; what they cannot do alone is say where it belongs, because that encodes whose pain matters more — the customer wrongly blocked or the fraud quietly paid. When Iris sits in the fraud-model review and asks "which do we need here, precision or recall — and how much of one are we trading for the other?", she is not visiting the data team's territory. She is doing her job at their table. The next page gives that conversation its full ledger.
- "Precision and recall are just two names for accuracy." They are two different questions with routinely different answers — the running example scores one half on precision and eight in ten on recall at the same moment. Accuracy is a third number again.
- "A good model simply maximizes both." Past a point they trade against each other through the threshold. The profession is choosing the right point on the trade — not pretending the trade away.
- "The threshold is a technical setting for the data team." It encodes whose pain matters more — blocked customers or unpaid fraud. That is a product and ethics decision, not a technical one, however technical the knob looks.
- "High precision means the model catches most of the fraud." Precision says nothing about coverage — a model that flags one obvious case a year has perfect precision and near-zero recall. Catching most of it is recall's department.
- These two words unlock every serious model-quality conversation you will sit in — and give you the question that actually matters: "precision or recall — which does our product need more?"
- The threshold dial from Topic 17 now has instruments attached: you can say what moving it buys and what it costs, in customer terms.
- Spam filter vs disease screening is a reusable compass — start any new classification discussion by asking which of the two mistakes is the expensive one.
Knowledge Check
The fraud detector flags 16 transactions; 8 turn out to be real fraud. There were 10 real frauds in total. What are its precision and recall?
- Precision 8 of 10; recall 8 of 16
- Precision 8 of 16; recall 8 of 10
- Both are 990 of 1,000, the same as accuracy
- Neither can be computed without knowing the threshold
Why is low recall called the "quiet" failure?
- Missed cases trigger no alarms — the damage arrives later, unannounced
- The model floods the team with so many alarms that they stop listening
- Recall is only measurable on models that run silently in the background
- Low recall barely matters, since the flagged cases were handled correctly
The team lowers the fraud threshold so the model flags far more transactions. What happens to precision and recall?
- Both rise — more flags mean more of everything good
- Recall falls while precision rises
- Recall rises while precision falls
- Nothing — the threshold affects speed, not these scores
Which product should lean precision-first, and which recall-first?
- Spam filter: recall-first. Disease screening: precision-first
- Both should maximize precision and recall equally at all times
- Neither — the balance is purely a technical choice made by the data team
- Spam filter: precision-first. Disease screening: recall-first
You got correct