Neurons and Layers

Seven chapters in, one kind of input has stayed politely out of reach. Plateful's churn table trains beautifully; the dish photos, the meaning of a review, a voice note to support — those defeated everything in Chapter 4. The machinery that finally cracked them is the neural network, and its reputation precedes it: brain-inspired, mysterious, the stuff AI headlines are made of. When the data team tells Iris the new photo checker is "a neural net", she braces for the one topic she assumes will be beyond her.

It isn't, and this page deflates it on purpose. A neural network is built from units so humble they barely earn the borrowed name neuron: each takes a few numbers in, multiplies each by a learned number, adds the results up, and passes one number on. That is the entire unit. The capability — and the word "deep" itself — comes from stacking thousands of them, and by the bottom of this page both words will be mechanical rather than magical.

One Neuron Is One Weighted Sum

Here is everything a neuron does. Inputs arrive as numbers — a delivery's distance and hour, or a patch of pixel values. Each input gets multiplied by its own weight — a learned number that says how much this input counts, and in which direction. The products are added into a single total, the total passes through a simple gate that decides how much of it flows onward, and that one number is the neuron's entire output. No memory, no judgment, no spark. A pocket calculator does more.

If the recipe sounds familiar, it should. Topic 19's linear model was exactly this: a weighted sum of features, with the weights as the learned parameters. A neuron is that model in miniature, and the resemblance is worth holding onto — the field's scariest-sounding building block is a device you already met three chapters ago. The weights are Topic 04's parameters under a new name; training will set them, and nothing else in the unit ever learns.

Layers: Sums of Sums

One weighted sum can only draw simple conclusions — Topic 19 was honest about that. The neural network's trick is composition. Neurons are arranged in layers: the first layer's neurons each weigh the raw inputs; the second layer's neurons weigh the outputs of the first layer; the third weighs the second. Every layer computes sums of sums, and combinations of combinations can express patterns that no single weighted sum could come close to — a claim you will watch pay off on the dish photos two pages from now.

Picture a company org chart processing one decision. Clerks each look at a few raw facts and pass up a one-line summary. Managers weigh the clerks' summaries and pass up their own. Directors weigh the managers'. Nobody in the building is brilliant — every person just weighs what lands on their desk — yet the hierarchy as a whole reaches conclusions no clerk could. That is a neural network: the intelligence, such as it is, lives in the arrangement, not in any unit. The clerks are the first layer, the directors the last, and the depth between them is where the network's reach comes from.

Trained by the Same Loop

How do all those weights get their values? By the loop you already own, unchanged. Start with random weights and absurd guesses. Guess, measure the miss — the loss from Topic 11 — nudge every weight slightly in the direction that shrinks it, and repeat. The only difference is bookkeeping: Topic 19's linear model had a weight per feature, maybe a few dozen; a serious network has millions of weights, every one of them nudged on every lap. The idea didn't get cleverer. The arithmetic bill exploded — hold that thought, because the next page is about who paid it.

And the finished product? Still Topic 04's file of numbers — just a much fatter file. Everything that page taught survives contact with deep learning: trained, then frozen, then used; confidently wrong remains possible; and a network with millions of weights can memorize its training examples instead of learning the pattern — Topic 14's overfitting disease, back with a bigger appetite. Deep learning is not an exception to the rules of this book. It is the rules, scaled up.

What Does "Deep" Actually Mean?

Now the term from Topic 02's map can finally be cashed in. Deep learning means machine learning with neural networks that have many layers. That's it — depth is a layer count. A network with two layers is shallow; one with dozens is deep; nothing about the word promises profundity, understanding, or depth of thought. It is an engineering measurement, like a building's height.

So the scariest term in the field unpacks into pieces you already own: weighted sums (Topic 19), stacked into layers, trained by the same guess-measure-nudge loop (Topic 11), producing the same frozen file of numbers (Topic 04), vulnerable to the same diseases (Topic 14). Which sharpens the real question rather than answering it: if the ideas are this simple — and they are decades old — why did deep learning only conquer the world recently? That is the next page.