Quality policy meme

You may remember I wrote about Quality Policies in a post last year. Then a couple of weeks ago, Kyle Chambers and I discussed them on his #QualityMatters podcast. Here I take one more pass, this time in meme format.

Feel free to let me know if you think I should stick to essays, and leave memes for people who are funnier and more creative ....  😀

Things change

A few weeks ago I was writing about the Context of the Organization (COTO), and there's one final point I'd like to make before I drop the subject. Your analysis is not a one-time exercise, because your context will change. Therefore you have to review the results and update them from time to time.

Image by Gerd Altmann from Pixabay

Intuitively this should make sense. Things change around you, and it's only natural that you will change to accommodate them. In 2019, only the smallest percentage of American companies had emergency plans in place to address a global pandemic; and the fraction of office jobs that could be done from home was tiny. By the dawn of 2021, every business in the country had figured out how to respond to a pandemic (regardless whether they had formalized the results in a document); and work-from-home had become a lot more common.

You can expect smaller changes, too. One of the outputs from your COTO analysis, after all, is a list of risks for your business. Then once you have a risk list, the basic principles of risk management say that you go to work addressing the most important ones: either take action to prevent them, or to mitigate them, or at least to define contingency actions after the fact in case one of them takes place.

But those very actions themselves now change the risk profile that you face. Maybe when you did your COTO-and-risk analysis you found five high-priority risks. But over the next few months you took steps to prevent two of them outright; you made two of the others a lot less likely (even if still possible); and you defined a recovery plan in case the last one happens. Are you still facing five high-priority risks? Of course not. The list has dropped to three, and (depending how you evaluate the likelihood and impact of those three after the measures you put in place) they might not all be high-priority any more.

Or you might have changed your company's strategy to focus on different products for a different market. Many years ago I worked for a small startup in the B2B space: we made products that we sold to other businesses. But one year we saw an opening and reoriented the company towards making smaller, individual-sized products for the home office market. That was some years before we got ISO 9001 certification, so we didn't have a written scope statement at the time. But if we'd had one, it would have changed. And COTO is directly tied to scope.

So set up a regular schedule to review and re-evaluate your COTO. It's simplest to make this part of your periodic Management Review, since the ISO 9001 requirements for Management Review include checking most of the elements of your COTO anyway. And then, based on the results of your review, propagate the changes to your risk lists and scope statement as well.

Bear in mind that it's a lot less work to revise your COTO than to set it up in the first place. Yes, things change; but mostly they don't all change at once. So it should be pretty simple to read through what you've already got and look for the places you have to edit.

__________

P.S.: This is probably not a burning concern of yours; but one consequence of the foregoing is that, since your QMS is based on your COTO and your COTO changes, there is no such thing as the perfect or final QMS. Everything can change, because everything depends on what you need—under changing conditions—in order to get what you want.    

          

The use and abuse of numerical models: a case study

This post is a little bit outside my normal range of topics, but it illustrates some important weaknesses in the use of mathematical models. Since we are all asked to create quantitative models from time to time, maybe the example is not out of place. I will ask your indulgence for the first couple of paragraphs, because it will take me a few minutes to get to the point where the models become important.

A number of days ago, Dawn Ringrose of Organizational Excellence Specialists mentioned on LinkedIn that she had discovered a recent podcast, and she encouraged those of us who follow her to listen to it. The podcast contains a deep and wide-ranging interview conducted by psychologist and public intellectual Jordan Peterson, where he talks with the authors Marian Tupy and Gale Pooley about their recent book Superabundance: The Age of Plenty. You can find the full recording here:

The basic thesis of the book is that abundance is increasing at a faster rate than the population, so over time we are all getting richer and richer—"so rich," in fact, that "we don't know how rich we are." A secondary thesis is that this abundance is driven by the application of human creative ingenuity, and therefore that we all benefit from (1) a large population of other people (more minds to create more new ideas) and also from (2) political and social forms that allow enough personal liberty that these ideas can be brought to market, where the price mechanism sorts out the good ones from the bad ones.

The negative argument

Thomas Malthus, by John Linnell - Gallery: 
Wellcome Collection gallery (2018-04-05): , CC BY 4.0 

It is important for me to clarify right at the beginning that the authors carry on two different kinds of arguments during their long discussion with Dr. Peterson, a negative one and a positive one. The negative argument is against Thomas Malthus, Paul Ehrlich, and a group that they characterize collectively as "radical environmentalists," "romantics," and "vicious anti-humanists." The positive argument is where they talk about the phenomenon of "superabundance" in the past, present, and future.

The negative argument is largely successful; its main drawback is that Tupy and Pooley spend so long at it that after a while I found it kind of exhausting. But even as I got tired of hearing them make the same points yet again, I had to agree that the points themselves were largely valid. The positive argument ... well, I'll talk about that in a minute.

What Malthus and Ehrlich and the others have in common is that they are alarmed at the growth of human population. And they express this alarm in clear predictions about the future—namely, that the continued growth of the worldwide human population will lead to widespread famine, malnutrition, disease, and warfare over an ever-decreasing pool of finite resources. The numerical calculations behind these predictions have always looked convincing, and so the arguments themselves have sounded plausible. Over the years, these arguments have been accepted by quite a lot of people. And they have led some agitated writers to describe humanity as "a cancer on the face of the earth."

The problem is that the apocalyptic predictions of Malthus and Ehrlich have never come to pass. Over the last hundred years (for example), worldwide levels of disease and malnutrition have dropped even as worldwide population has sharply increased. The prices of many resources have fallen. In short, results in the real world have consistently contradicted the results predicted by the mathematical models used by Malthus and Ehrlich. This proves that those models were wrong.

One drawback of repeating the negative arguments so often is that sometimes it is easy to confuse the question whether Tupy and Pooley are refuting their adversaries or making a positive case in their own name. But the distinction is important. Even though Malthus and Ehrlich were certainly wrong, that doesn't ipso facto prove that Tupy and Pooley are right. Still, as I say, the negative argument is largely successful. 

The positive argument

The positive argument uses extensive historical research and careful mathematical modeling to show that prices have gotten cheaper over the long haul, with the result that individuals have gotten (in terms of buying power) correspondingly richer. Specifically, Tupy and Pooley identify fifty commodities spanning a wide range of use-cases, and show that over the long term the prices for all of them have dropped steadily and dramatically.

Comparing prices across a century or more can be a challenge, because the value of one dollar in 1822 has more or less no relation at all to the value of a dollar in 1922 or 2022. Tupy and Pooley have an ingenious solution for this problem. Instead of looking at prices as marked on a price tag—this is called "prices in nominal dollars"—they ask, "How long would a person in that year have had to work in order to earn enough money to buy this or that? And how long would a person today have to work to buy the same thing?" They call this the time-price of a good, and so the foundation of their research is a study of time-prices across the years for fifty different commodities. Naturally one risk in the calculation of a time-price is that you have to use a standard rate of pay and different jobs pay different wages. The standard they choose is the average rate of pay for blue-collar or unskilled labor. Their goal is not to show that life has gotten better for millionaires, after all, but that it has gotten better for everyone.

So, as I say, the foundation of their research is a tabulation of the time-prices of a wide range of commodities across the years. What they show is that these prices have fallen consistently over time, and keep falling. Their conclusion is to reject the narrative that says humanity is confronted with finite resources. Finite resources, they say, are just a myth. Based on their data, they argue, all commodities are getting cheaper and more abundant.

Real-world results

But in fact not all commodities can be made infinitely abundant. Consider the commodity "Real estate within 50 miles of downtown Los Angeles." There is a theoretical maximum of π x 50 x 50 = about 7854 square miles of such real estate—the actual number is a little over half that because the rest is covered by the Pacific Ocean. Call it maybe about 4000 square miles, of which some is too mountainous to build on. But in any event, when it is gone it is gone. The price mechanism cannot create more land within 50 miles of downtown Los Angeles, once it has all been built on. The classic answer from economists is that when one resource becomes unavailable, a high price will encourage substitution of a similar good instead. And yes, I'm sure that when all the land within 50 miles of downtown Los Angeles is built up, there will still be land available within 50 miles of downtown Bozeman, Montana. Whether that constitutes an adequate substitute is perhaps a matter of taste; I know some people who would actively prefer Bozeman to LA. But moving to southern Montana instead of Southern California will at any rate make it hard to drive into town to catch an evening performance at the Ahmanson.

These theoretical limits have real-world consequences. My parents bought their house in 1973. Even after you adjust for inflation—in other words, measuring in constant dollars—that house now costs 3.5 times what it cost when they bought it. What about the time-price? Remember that Tupy and Pooley express the price of commodities in terms of the number of hours of unskilled labor it costs to acquire them, so let's do that. In 1973 a laborer or helper in the building trades made $6.06 per hour. The equivalent pay in 2022 is $15.00 per hour. Looking at my parents' house, then, in 1973 it cost 5776 hours of unskilled labor to buy it in cash. The same house in 2022 costs 55,260 hours of unskilled labor to buy it in cash. In other words, measured in time-price of unskilled labor, the price of that same house today is nearly ten times what it was in 1973! So much for the claim that everything is getting cheaper.

Or what about medical care? I recently found an article on the Internet reporting a hospital bill from 1961 to deliver a baby: total cost before insurance was $419. Today that number is $18,865. Measuring in time-price instead of nominal dollars, we calculate that the hospital stay for delivering a baby cost just over 69 hours of unskilled labor in 1961. In 2022, the same delivery costs over 1257 hours of unskilled labor, or more than 18 times as much.

In other words, when Tupy and Pooley chose what to measure, that choice affected the outcome of their calculations. Does it matter, or am I just cherry-picking examples to make them look bad? It's a fair question, so let's step back from the numbers for a minute to take an overall look at the big picture. 

• In 1960, a man earning one full-time blue-collar salary could buy a house and raise a family. Detroit was full of these men; so were countless towns spread across the industrial Northeast. 
• Today a man earning one full-time blue-collar salary can do neither of those things, if he can find a job at all. This is part of why the once-thriving Northeast is now called "the Rust Belt."

These real-life consequences are the direct opposite of what the "superabundance" model predicts. And when results in the real world contradict the results predicted by a model, that proves that the model is wrong. It fails to match reality. The Tupy and Pooley "superabundance" model is, in its way, just as wrong—and thereby just as flawed and just as pernicious—as the models used by Malthus and Ehrlich.

Why does their argument sound so plausible? Well, Malthus's argument sounded plausible too. So did Ehrlich's. These things are complicated, they have a lot of different aspects, and getting them right is really hard. Numerical models can make the picture simpler, so we have a better chance to understand it. But numerical models can distort as much as they illuminate, even if unintentionally. You can find a quick summary of some of the traps that threaten the unwary in, for example, Darrell Huff's delightful little book How to Lie With Statistics. Or consider the remark in an earlier post in this blog that "There is no metric in the world that cannot be gamed." If you're not careful, there is always the risk that your numbers will prove anything.

To bring this back to the Quality profession, the critical point is that you can never trust the numbers in any model without checking them both quantitatively and qualitatively against the real world. Or more memorably, "However beautiful the strategy, you should occasionally look at the results."* 

To summarize my assessment of the initial podcast, their negative argument is fine but they spend too much time on it. Their positive argument goes astray, and ends up wrong. A lot of the people who logged into YouTube to leave comments on this conversation were really impressed by it, but in the end I'm not one of them.

__________

* This phrase has been attributed to Winston Churchill, though the attribution has also been disputed.                     

Quality policies of warm vanilla pudding

Once again I had a chance to sit down with the exuberant Kyle Chambers of Texas Quality Assurance for a wide-ranging conversation about Quality Policies and what's wrong with so many of them. Over the course of the discussion we explore what policies are really for, how the ISO 9001 standard makes the situation worse, and why you might write your policies differently in Texas and California. 

Please join us!

You can find the podcast version here: #QualityMatters episode 149.

Or there's a version on YouTube that also includes video, which you can find here:

Leave me a comment to let me know your thoughts!

      

Performance art? Or just stupid rules?

Image by Bernd Hildebrandt from Pixabay

A couple of days ago—well, it will be more by the time this post is published—Chris Paris wrote a post arguing that certification audits have become a kind of "performance art." He illustrated his point with a schedule one of his clients had gotten from their certification body, and walked his readers through it to show just how much padding and non-value-adding time it contained. 

I have to wonder, though: Was this artifact really a confession that the whole audit process is just a stylized drama empty of meaning? Was it—as Paris suggests—"essentially B2B theft" [emphasis in the original] because the client was paying for a full audit without getting one? Or was the auditor just responding in the best way he could to a set of inflexible rules set by the certification body concerning audit duration?

Of course there have to be rules about audit duration, and at some level they have to be inflexible. Otherwise every single client will argue "But we're a special case," and there will be no consistency from one audit to the next. The risk of purely subjective results would be even greater than it is today.

But it is simply not possible to write meaningful rules that cover every contingency. Any system of rules will always paint with a broad brush, and so there will always be cases that don't fit. Paris's client is clearly one of them: while the rules calculated that a certain number of hours were mandatory, Paris's own description of his client's business makes it obvious that spending the mandatory number of hours in meaningful audit would have been far, far too much.

To be clear, I don't know the details about Paris's client or their CB. Maybe in this case everything Paris says against the CB is justified. But it's not always so.

For example, I've seen something similar in my own experience. I once worked for a Business Unit that created a special location for its worldwide headquarters, half an hour's drive from an already-established regional headquarters. There must have been reasons, but those reasons were never clear to most of us. Sitting in this "worldwide headquarters" was a small software team who just happened to be in the same place anyway, and then the heads of all the global departments. But their respective staffs were distributed around the world in the company's regular offices. This one location had the Global BU President, all the global BU Vice-Presidents, a couple of administrative assistants, and (as I mentioned) a small software team who had no particular connection with anyone else there.

When it came time to schedule our audits, our CB told us this site required three full audit days. I asked why, and they showed me the calculation: because we were in this line of work, we had to use that table; because the overall organization was this size, we had to use that row in the table; then there was a factor because this site did office work (production would have involved a different factor); and finally there was a factor because this site was the Global Headquarters. Grand total: 3.0 audit-days. End of discussion.

I hope it's obvious from my description of who was actually located in this site that there was no way we could stretch the audit to three days. I worked with the auditor directly, and between us we came up with a schedule that the CB approved. But in reality he and I spent a lot of time sitting in the conference room alone, just talking. And we allocated a lot of time for lunch. It was a waste of time and money, but trying to enforce three full days of genuine auditing would have been far more wasteful.

I think the following year I told the CB we had "moved" the headquarters to the big office half an hour away. There was lots going on in that office, and we never ran short of functions to audit.

In principle, the best answer would be for the rules that determine audit-days to account for all possible variations, so that we don't run into the kind of foolishness that Paris and I have both described. But I don't think we can ever do that. The real word will always be more complex and full of variety than any set of rules we can devise. So long as that's the case, it's hard for me to get too exercised when the principals have to adapt the rules to fit reality. The Duc de La Rochefoucauld reminds us, after all, that "Hypocrisy is the homage vice pays to virtue"; and sometimes a small homage can smooth over difficulties more easily than a full corrective action.