The Wisdom of Crowds
Iris finally asks which model actually powers Plateful's delivery estimate — the tree? the linear one? — and gets an answer that sounds like a joke: it isn't one model. It is three hundred small decision trees, each making its own guess, and the app shows their average. The team's best model is a committee.
Combining many mediocre models into one strong one is called an ensemble, and it is among the most reliable tricks in all of machine learning — the actual engine behind a huge share of production predictions on business tables. This page explains why a crowd of models beats its own members, meets the two classic ensemble designs, and names the price the crowd charges.
Why Committees Beat Soloists
The idea is older than computers. At an English county fair in 1906, nearly 800 visitors paid to guess the weight of an ox. Individually they were all over the place — hundreds of pounds off in both directions. But the average of all the guesses came out within a single pound of the ox's true weight, closer than almost any individual, including the cattle experts. The crowd as a whole knew something no member knew.
The mechanism is the key, because it transfers to models exactly. Each fairgoer erred, but they erred differently — one too high, another too low, each misled by their own angle — so in the average, the errors partly cancelled and the shared signal remained. The same holds for models: a committee helps only when its members are differently wrong. Average three hundred copies of the same model and you get the same answer with the same mistakes, three hundred times. Diversity of errors, not headcount, is what does the work — hold onto that as the analogy retires and the term takes over: an ensemble earns its power from diverse members.
The Random Forest — Many Trees, One Vote
So how do you get three hundred usefully different models out of one dataset? The classic answer builds diversity in on purpose. Train hundreds of decision trees — the very trees from the last page — but give each one only a random slice of the data: a random sample of the rows, and at each question, a random subset of the features to choose from. Every tree grows up with a different partial view, so every tree errs in its own way. Then combine them: average their numbers for regression, count their votes for classification. The name is unusually honest — a random forest: randomness for diversity, a forest of trees for the crowd.
Any single tree in that forest is mediocre on purpose — shallow, half-blind, trained on a fraction of the past. The forest is the reliably strong one. This is the model that beat everything else the data team tried on delivery times, and it will carry that title through the rest of the book: when Chapter 7 walks a model into production and Chapter 8 asks whether deep learning could do better, the delivery model they're talking about is this forest.
Boosting — a Relay of Error-Fixers
The forest builds its committee all at once, in parallel. The other classic ensemble, boosting, builds it as a relay. Train one small model; look at which examples the committee-so-far gets wrong; train the next small model to focus on exactly those mistakes; repeat, hundreds of times, each new member a specialist in the errors of everyone before it. The final prediction combines the whole relay. That is the honest one-paragraph version — the internals can wait for a hands-on course. What's worth knowing now is that boosted ensembles trade blows with random forests for the title of best model on tabular business data, and you will hear their tool names (XGBoost is the famous one) in any data team's kitchen.
The Price of the Crowd
Nothing this effective comes free, and the bill arrives in two currencies. The first is resources: three hundred trees cost more to train, more to store, and more to run on every single prediction than one tree does. Usually that is fine — the trees are small and computers are fast — but "usually" is a judgment someone has to make against real latency and hardware budgets.
The second currency is the expensive one: readability. The last page celebrated the single tree because you could follow its reasoning end to end — far, rush hour, 52 minutes, done. Nobody reads three hundred trees. The forest gives a better answer and a worse explanation, and that is a genuine trade with a name worth remembering: accuracy versus readability. You met it here in miniature; Chapter 10 makes it serious, when the person asking "why?" is a customer the model turned down. On business tables the trade usually favors the crowd — but it should be chosen, not stumbled into.
- "An ensemble is just running one model several times." The same model repeated errs identically, and identical errors don't cancel. The power comes from members that are differently wrong — diversity is built in on purpose.
- "More models in the committee is always better." Gains flatten while cost keeps growing. Past a point, tree number 301 adds expense, not accuracy — diversity, not headcount, does the work.
- "Random forests are unrelated to decision trees." A forest is literally many trees — the last page's model, industrialized. For once, the name says exactly what the thing is.
- "The forest can explain its answer the way a single tree can." It can't — no one reads three hundred trees. The lost readability is precisely the price paid for the crowd's accuracy.
- "It's a random forest" is one of the most common sentences in applied ML — and it now unpacks itself: many trees, random slices, one vote.
- The accuracy-versus-readability trade you met here is your first taste of a theme Chapter 10 makes serious — better answers can cost you the ability to say why.
- Knowing that diversity, not headcount, powers a committee gives you a sharp question for any ensemble claim: "what makes the members err differently?"
Knowledge Check
Why can a committee of mediocre models beat a single strong one?
- Because each individual member model is already more accurate on its own
- Because members erring differently cancel each other's mistakes
- Because many models can compute the answer faster together
- Because the models share what they learned with each other
What is a random forest?
- Many decision trees, each on a random slice, answering by vote or average
- One very large decision tree with thousands of questions
- The same decision tree retrained many times until it stops making mistakes
- An algorithm that picks a random model for every prediction
How does boosting build its committee?
- All members train at once, each on a random slice of the data
- Each new model is bigger and more complex than the one before
- Each new model trains on what the committee so far gets wrong
- The best single model is copied hundreds of times and averaged
Plateful replaces its single delivery tree with the 300-tree forest. What does the team give up?
- Resistance to overfitting — since big committees end up memorizing the noise
- The training data — forests need a different kind of dataset
- Readability — plus some speed and cost, for better predictions
- The ability to predict numbers — forests only classify
You got correct