Chapter Ten · Where Models Meet People

ML in the Real World

Nine chapters judged models by their predictions. This chapter judges them by their consequences. When a model scores a customer, ranks a courier, or learns from your data, the question stops being "is it accurate?" and becomes "is it fair, is it private, can it explain itself, what can't it see — and what does it do to the work we get paid for?" Five topics, one honest tour of where the machinery touches people.

5 topics

Everything so far happened inside the machinery: data in, training, metrics, deployment, generation. But Plateful's models were never really pointed at dashboards — they were pointed at people. The courier who gets the good dinner slot or doesn't. The customer whose behavior became training data without anyone quite asking. The loyal regular declined a payment option with no reason given. Each of those is a person on the receiving end of a file of numbers, and each one raises a question no accuracy score can answer.

This chapter asks those questions in order. How bias gets into a model no one intended to bias — and what it has already done out in the world. What training on people's data means for privacy, from both seats you sit in. Why "the computer said no" is not an acceptable sentence for decisions that matter. The two limits baked into every prediction: correlation is not causation, and deployed predictions reshape the world they predict. And finally the question Iris gets at family dinners — what all this does to jobs — answered evenly, without utopia or doom.

One model, and everyone it touches — the questions of this chapter
Plateful's courier-assignment model
The courier & the customeris it fair to them?can it say why?
The data it learned fromdid people consent?
The regulatoris it accountable?
The companywhat are its limits?what happens to the work?

Topics in This Chapter