Garbage In, Garbage Out
In a planning meeting, Iris pitches an ML feature she's proud of, and the data team turns it down with four words: "we'd learn the wrong thing." Not "the model isn't strong enough" or "we'd need more budget" — the examples themselves would teach a falsehood, so nothing built on them could be right. With that, Iris meets the oldest law of the field, older than the field itself: garbage in, garbage out.
The law says a model can only be as good as its examples. Not as good as the algorithm, not as good as the budget, not as good as the ambition — the examples. The previous page showed data that is visibly broken: gaps, duplicates, nonsense. This page is about the sharper edge of the same law: data that is perfectly clean and still wrong, because it shows the model only part of the world.
The Ceiling Is the Data
Training — the subject of the next chapter — extracts patterns from the table. That sentence contains the whole law: it extracts patterns from the table. A pattern present in the examples can be found; a pattern absent from them cannot be learned at any price. The table sets a ceiling, and no amount of skill above it changes where it hangs.
This is why "use a better algorithm" is the wrong reflex when the data is wrong. A fancier algorithm learns the same wrong patterns — at best more efficiently. If the examples teach a falsehood, the deluxe model learns the falsehood with more polish. Effort spent above the ceiling raises cost, not truth; the ceiling only moves when the examples do.
Blind Spots: Data That Misses Part of the World
Here is the trap in its most common form. Plateful's delivery data describes the customers Plateful already has — their districts, their restaurants, their habits. Now the company launches in a new district. The model happily predicts delivery times there, with its usual confidence, about streets and customers it has never seen a single example of. The data was clean. The predictions are nonsense.
This kind of gap has a name: sampling bias — when the collected examples systematically miss or under-represent part of the world the model will be used on. The word "bias" here doesn't accuse anyone of prejudice; it describes a table that leans, the way a poll taken only in one neighborhood leans. And it's treacherous precisely because it is invisible from inside the table: no column records who isn't in it. Every check the team runs on the data it has will pass; the flaw lives in the data it doesn't have.
Old Data, Old World
There is a second way clean data can lie: by being true about a world that has ended. Examples are records of the past, and the patterns in them are the past's patterns. If Plateful redrew its delivery zones, raised prices, or watched customer habits shift since the data was recorded, the model doesn't notice — it keeps faithfully predicting the world as it was. The predictions look as confident as ever; only reality has moved.
Both failures fit one everyday picture: studying for an exam from last year's papers. If half of this year's topics never appeared in past papers, no amount of memorizing covers them — that's the blind spot. And if the syllabus changed since last year, perfect memorization prepares you flawlessly for the wrong exam — that's stale data. Coverage and freshness: the two questions last year's papers can't answer for you, and the two questions a training table must. (Models going stale in production has a formal name and its own machinery — Chapter 7 picks it up as drift.)
The Right First Question
All of this compresses into one professional habit. Before "which model should we use?", the pros ask: does our data actually contain the answer — for everyone we'll use it on? It is the cheapest question in machine learning — it needs no computers, no training runs, no budget — and it is the most skipped, because "which model?" sounds like progress and "is our table telling the truth?" sounds like doubt.
It's also the question with the longest shadow. When the people a dataset under-represents are groups of customers — a district, an age group, a language community — a blind spot stops being a technical nuisance and becomes a model that quietly serves some people worse than others; Chapter 10 is built on exactly this foundation. For now, close the chapter with its one-sentence summary: the data is the raw material, and no downstream brilliance improves the ore.
- "A better algorithm can compensate for bad data." It learns the same wrong patterns more efficiently. The examples set a ceiling, and algorithms live under it.
- "Lots of data means representative data." Ten million rows about one district still know nothing about the next one. Volume raises confidence; only coverage raises truth.
- "Bias in data means someone was prejudiced." Usually it's mechanical — whoever happened to get recorded is who the model knows. No villain required, which is exactly why it slips through. (The human impact gets its own chapter — Chapter 10.)
- Most real-world ML failures you will read about trace back to this page — wrong or partial examples, not evil algorithms. You now have the diagnosis before the headlines hand it to you.
- Chapter 10's fairness chapter stands on this one: when a model treats people unfairly, the unfairness usually walked in through the training table.
Knowledge Check
The team doubles the budget and switches to a much fancier algorithm, keeping the same flawed data. What improves?
- Accuracy — a stronger algorithm can reason past the flaws in the table
- Coverage — a bigger model fills in the parts of the world the data missed
- Very little — it learns the same wrong patterns, just more efficiently
- Everything — algorithm choice is the main limit on model quality
Plateful's model was trained entirely on city-center orders, then launched in the suburbs — where it predicts badly. What is this?
- Messy data — the training rows must contain typos and duplicates
- Sampling bias — the examples missed the world the model serves
- A labeling failure — humans marked the suburban answers wrong
- Stale data — the world changed between training and launch
Plateful has ten million clean rows about Riverside and none about the newly launched district. Does the sheer volume help?
- Yes — past a few million rows, a model generalizes to anywhere
- Yes, as long as the rows are clean — clean data is representative data
- No, because ten million rows is too many to train on
- No — more rows about one district teach nothing about another
Plateful redrew its delivery zones and raised prices last spring, but the model still runs on data from before the change. What should Iris expect?
- Confident predictions of a world that no longer exists
- The model refuses to answer, flagging its data as out of date
- The model gradually adapts as new orders come in
- Random answers with visibly low confidence on every order
Does "sampling bias in the data" mean someone at Plateful acted with prejudice?
- Yes — biased data always starts with a biased person choosing what to record
- Yes, and Chapter 10 explains how to trace who was responsible
- Not necessarily — often it's mechanical: recorded is what the model knows
- No — bias is a math error that happens inside the training loop
You got correct