A Tour of Simple Models

Iris keeps hearing model names dropped in meetings as if they were characters in a show everyone else has watched. "We tried a decision tree first." "The linear model still wins on that one." "Honestly, nearest neighbors would get you halfway." Until now this book has treated the model as a sealed box of learned numbers. Time to open three of the boxes — the three simplest — and look at how each one actually turns features into an answer.

No formulas are coming; the math implements these ideas, it doesn't replace them. The point of the tour is smaller and more useful: different models are different shapes of reasoning. Picture three colleagues estimating a delivery. One runs down a checklist of questions. One keeps a tariff card — so much per kilometer, so much for rush hour — and adds it up. One shrugs and says "the last five orders like this took about forty". All three are legitimate strategies, and each one is a real model family.

The Decision Tree — a Chain of Questions

A decision tree is the checklist colleague. It answers by asking a sequence of learned yes/no questions about the features, and each answer decides which question comes next, until the chain ends in a prediction. The crucial word is learned: nobody writes the questions. Training discovers, from the labeled examples, which question splits the data most usefully first, which one next, and what number each path should end in.

Here is a real one — tiny, but genuinely the shape the machine produces — for Plateful's delivery time. Read it top to bottom and notice you can follow every step of its reasoning:

That readability is the tree's signature talent. When the model predicts 52 minutes, anyone — Iris, a support agent, a courier — can see exactly why: far away, rush hour, done. Keep that property in mind; Chapter 10 will make "can we read its reasoning?" a serious question, and the tree is its champion.

The Linear Model — a Weighted Sum

The linear model is the tariff-card colleague. Instead of asking questions, it gives every feature a learned weight — a price, in effect — and adds the bill up. Start from a base of 18 minutes. Add 3 minutes per kilometer. Add 6 if it's rush hour. Add 4 if the restaurant has been slow this week. Total: the prediction. Different features, different weights — but always one flat sum.

Those weights should sound familiar: they are the parameters from Topic 04, the numbers in the model file — except here there are few enough that you can print them on a card and read them. A weight of +3 minutes per kilometer isn't buried inside anything; it is the model's entire opinion about distance, stated in plain units. That makes the linear model the other human-readable one on this page, and the tree's only rival for Chapter 10's affections.

k-Nearest Neighbors — Ask the Similar Past

The third colleague barely builds a model at all. k-nearest neighbors answers a new case by finding the k most similar past examples — say, the 5 old orders closest to this one in distance, hour, and restaurant — and answering whatever happened to them: their average minutes for regression, their majority kind for classification. The "k" is just how many neighbors get a say.

There is no training loop here worth the name — the method's knowledge is the stored table of past examples, consulted fresh for every question. That sounds too naive to work, and yet on the right problem it is embarrassingly effective: when similar inputs genuinely lead to similar outcomes, simply asking the similar past is hard to beat. Its weakness is the flip side — it must search the past for every single prediction, which gets slow as the table grows, and "similar" itself is only as good as the features.

Why Simple Survives

Three strategies, one recap before the point of the page:

None of these is yesterday's news. Simple models remain the workhorses behind a huge share of real business predictions, and professionals reach for them first — deliberately. They train in seconds, cost almost nothing to run, and their reasoning can be inspected when something looks off. And recall Topic 14: a model can only memorize as much noise as it has capacity for. A three-level tree or a one-line sum has nowhere to hide the noise, which makes simple models stubbornly hard to overfit on small data.

So the professional move is to start simple and let the data argue for more. If the simple model already predicts well, the project is done early and cheaply. If it doesn't, its misses show what the extra complexity must earn. Chapter 8 completes this arc — including why, on ordinary business tables, the simple-and-medium models often beat the deep learning that dazzles everywhere else.