The use and abuse of numerical models: a case study

This post is a little bit outside my normal range of topics, but it illustrates some important weaknesses in the use of mathematical models. Since we are all asked to create quantitative models from time to time, maybe the example is not out of place. I will ask your indulgence for the first couple of paragraphs, because it will take me a few minutes to get to the point where the models become important.

A number of days ago, Dawn Ringrose of Organizational Excellence Specialists mentioned on LinkedIn that she had discovered a recent podcast, and she encouraged those of us who follow her to listen to it. The podcast contains a deep and wide-ranging interview conducted by psychologist and public intellectual Jordan Peterson, where he talks with the authors Marian Tupy and Gale Pooley about their recent book Superabundance: The Age of Plenty. You can find the full recording here:

The basic thesis of the book is that abundance is increasing at a faster rate than the population, so over time we are all getting richer and richer—"so rich," in fact, that "we don't know how rich we are." A secondary thesis is that this abundance is driven by the application of human creative ingenuity, and therefore that we all benefit from (1) a large population of other people (more minds to create more new ideas) and also from (2) political and social forms that allow enough personal liberty that these ideas can be brought to market, where the price mechanism sorts out the good ones from the bad ones.

The negative argument

Thomas Malthus, by John Linnell - Gallery: 
Wellcome Collection gallery (2018-04-05): , CC BY 4.0 

It is important for me to clarify right at the beginning that the authors carry on two different kinds of arguments during their long discussion with Dr. Peterson, a negative one and a positive one. The negative argument is against Thomas Malthus, Paul Ehrlich, and a group that they characterize collectively as "radical environmentalists," "romantics," and "vicious anti-humanists." The positive argument is where they talk about the phenomenon of "superabundance" in the past, present, and future.

The negative argument is largely successful; its main drawback is that Tupy and Pooley spend so long at it that after a while I found it kind of exhausting. But even as I got tired of hearing them make the same points yet again, I had to agree that the points themselves were largely valid. The positive argument ... well, I'll talk about that in a minute.

What Malthus and Ehrlich and the others have in common is that they are alarmed at the growth of human population. And they express this alarm in clear predictions about the future—namely, that the continued growth of the worldwide human population will lead to widespread famine, malnutrition, disease, and warfare over an ever-decreasing pool of finite resources. The numerical calculations behind these predictions have always looked convincing, and so the arguments themselves have sounded plausible. Over the years, these arguments have been accepted by quite a lot of people. And they have led some agitated writers to describe humanity as "a cancer on the face of the earth."

The problem is that the apocalyptic predictions of Malthus and Ehrlich have never come to pass. Over the last hundred years (for example), worldwide levels of disease and malnutrition have dropped even as worldwide population has sharply increased. The prices of many resources have fallen. In short, results in the real world have consistently contradicted the results predicted by the mathematical models used by Malthus and Ehrlich. This proves that those models were wrong.

One drawback of repeating the negative arguments so often is that sometimes it is easy to confuse the question whether Tupy and Pooley are refuting their adversaries or making a positive case in their own name. But the distinction is important. Even though Malthus and Ehrlich were certainly wrong, that doesn't ipso facto prove that Tupy and Pooley are right. Still, as I say, the negative argument is largely successful. 

The positive argument

The positive argument uses extensive historical research and careful mathematical modeling to show that prices have gotten cheaper over the long haul, with the result that individuals have gotten (in terms of buying power) correspondingly richer. Specifically, Tupy and Pooley identify fifty commodities spanning a wide range of use-cases, and show that over the long term the prices for all of them have dropped steadily and dramatically.

Comparing prices across a century or more can be a challenge, because the value of one dollar in 1822 has more or less no relation at all to the value of a dollar in 1922 or 2022. Tupy and Pooley have an ingenious solution for this problem. Instead of looking at prices as marked on a price tag—this is called "prices in nominal dollars"—they ask, "How long would a person in that year have had to work in order to earn enough money to buy this or that? And how long would a person today have to work to buy the same thing?" They call this the time-price of a good, and so the foundation of their research is a study of time-prices across the years for fifty different commodities. Naturally one risk in the calculation of a time-price is that you have to use a standard rate of pay and different jobs pay different wages. The standard they choose is the average rate of pay for blue-collar or unskilled labor. Their goal is not to show that life has gotten better for millionaires, after all, but that it has gotten better for everyone.

So, as I say, the foundation of their research is a tabulation of the time-prices of a wide range of commodities across the years. What they show is that these prices have fallen consistently over time, and keep falling. Their conclusion is to reject the narrative that says humanity is confronted with finite resources. Finite resources, they say, are just a myth. Based on their data, they argue, all commodities are getting cheaper and more abundant.

Real-world results

But in fact not all commodities can be made infinitely abundant. Consider the commodity "Real estate within 50 miles of downtown Los Angeles." There is a theoretical maximum of π x 50 x 50 = about 7854 square miles of such real estate—the actual number is a little over half that because the rest is covered by the Pacific Ocean. Call it maybe about 4000 square miles, of which some is too mountainous to build on. But in any event, when it is gone it is gone. The price mechanism cannot create more land within 50 miles of downtown Los Angeles, once it has all been built on. The classic answer from economists is that when one resource becomes unavailable, a high price will encourage substitution of a similar good instead. And yes, I'm sure that when all the land within 50 miles of downtown Los Angeles is built up, there will still be land available within 50 miles of downtown Bozeman, Montana. Whether that constitutes an adequate substitute is perhaps a matter of taste; I know some people who would actively prefer Bozeman to LA. But moving to southern Montana instead of Southern California will at any rate make it hard to drive into town to catch an evening performance at the Ahmanson.

These theoretical limits have real-world consequences. My parents bought their house in 1973. Even after you adjust for inflation—in other words, measuring in constant dollars—that house now costs 3.5 times what it cost when they bought it. What about the time-price? Remember that Tupy and Pooley express the price of commodities in terms of the number of hours of unskilled labor it costs to acquire them, so let's do that. In 1973 a laborer or helper in the building trades made $6.06 per hour. The equivalent pay in 2022 is $15.00 per hour. Looking at my parents' house, then, in 1973 it cost 5776 hours of unskilled labor to buy it in cash. The same house in 2022 costs 55,260 hours of unskilled labor to buy it in cash. In other words, measured in time-price of unskilled labor, the price of that same house today is nearly ten times what it was in 1973! So much for the claim that everything is getting cheaper.

Or what about medical care? I recently found an article on the Internet reporting a hospital bill from 1961 to deliver a baby: total cost before insurance was $419. Today that number is $18,865. Measuring in time-price instead of nominal dollars, we calculate that the hospital stay for delivering a baby cost just over 69 hours of unskilled labor in 1961. In 2022, the same delivery costs over 1257 hours of unskilled labor, or more than 18 times as much.

In other words, when Tupy and Pooley chose what to measure, that choice affected the outcome of their calculations. Does it matter, or am I just cherry-picking examples to make them look bad? It's a fair question, so let's step back from the numbers for a minute to take an overall look at the big picture. 

• In 1960, a man earning one full-time blue-collar salary could buy a house and raise a family. Detroit was full of these men; so were countless towns spread across the industrial Northeast. 
• Today a man earning one full-time blue-collar salary can do neither of those things, if he can find a job at all. This is part of why the once-thriving Northeast is now called "the Rust Belt."

These real-life consequences are the direct opposite of what the "superabundance" model predicts. And when results in the real world contradict the results predicted by a model, that proves that the model is wrong. It fails to match reality. The Tupy and Pooley "superabundance" model is, in its way, just as wrong—and thereby just as flawed and just as pernicious—as the models used by Malthus and Ehrlich.

Why does their argument sound so plausible? Well, Malthus's argument sounded plausible too. So did Ehrlich's. These things are complicated, they have a lot of different aspects, and getting them right is really hard. Numerical models can make the picture simpler, so we have a better chance to understand it. But numerical models can distort as much as they illuminate, even if unintentionally. You can find a quick summary of some of the traps that threaten the unwary in, for example, Darrell Huff's delightful little book How to Lie With Statistics. Or consider the remark in an earlier post in this blog that "There is no metric in the world that cannot be gamed." If you're not careful, there is always the risk that your numbers will prove anything.

To bring this back to the Quality profession, the critical point is that you can never trust the numbers in any model without checking them both quantitatively and qualitatively against the real world. Or more memorably, "However beautiful the strategy, you should occasionally look at the results."* 

To summarize my assessment of the initial podcast, their negative argument is fine but they spend too much time on it. Their positive argument goes astray, and ends up wrong. A lot of the people who logged into YouTube to leave comments on this conversation were really impressed by it, but in the end I'm not one of them.

__________

* This phrase has been attributed to Winston Churchill, though the attribution has also been disputed.