"What gets measured gets managed"—like it or not!

For the past couple of weeks we've been talking about metrics, and it is clear that they are central to most modern Quality systems. ISO 9000:2015 identifies "Evidence-based decision making" as a fundamental Quality management principle, stating (in clause 2.3.6.1): "Decisions based on the analysis and evaluation of data and information are more likely to produce desired results." ISO 9001:2015 (in clause 6.2.1) requires organizations to establish measurable quality objectives—that is, metrics—in order to monitor how well they are doing. We've all heard the slogan, "What gets measured, gets managed."

If you think about it, the centrality of quantitative metrics relies on a number of fundamental assumptions:

• We assume that quantitative metrics are objective—in the sense that they are unbiased. This lack of bias makes them better than mere opinions.
• We also assume that quantitative metrics are real, external, independent features of the thing we want to understand. This external independence makes them reliable as a basis for decisions. 
• And finally, we assume that quantitative metrics are meaningful: if the numbers are trending up (or down), that tells us something about what action we need to take next.

But each of these assumptions is weak.

• Metrics are not necessarily unbiased. In fact, as we discussed last week, there is a sense in which every quantitative metric conceals some hidden bias. Since this is true for all metrics, the answer is not to replace your old metric with a better one. What is important is to understand the hidden bias, to correct for it when you interpret your results. 
• Metrics are not necessarily external or independent of the thing being measured. Think about measuring people. If they come to understand that you are using a metric as a target—maybe they get a bonus if the operational KPIs are all green next quarter—people will use their creativity to make certain that the KPIs are all green regardless of the real state of things. (See also this post here.)
• And metrics can only be meaningful in a defined context. Without the context, they are just free-floating numbers, no more helpful than a will o' the wisp. 

We discussed the first risk last week. I'll discuss the second risk in this post. And I'll discuss the third one next week. 

Unhelpful optimization

I quoted above the slogan, "What gets measured, gets managed." But just a week ago, Nuno Reis of the University of Uncertainty pointed out in a LinkedIn post that this slogan is misleading, and that it was originally coined as a warning rather than an exhortation. Specifically, Reis writes:

It started with V. F. Ridgway’s 1956 quote: "What gets measured gets managed."

Yet, Ridgway was WARNING how metrics distort and damage organizations.

The FULL quote is:

"What gets measured gets managed—even when it's pointless to measure and manage it, and even if it harms the purpose of the organization to do so."*

The original source was a 1956 article by V. F. Ridgway called. "Dysfunctional consequences of performance measurements."** Ridgway's point is that a metric provides just a single view onto the thing you want to understand, but some people will always treat it uncritically, as the whole truth. This misunderstanding creates an opportunity for other people to exploit the metric by acting so that the numbers get better, even if the overall organization suffers for it. Examples include the following:***

"1. A case where public employment interviewers were evaluated based on the number of interviews. This caused the interviewers to conduct fast interviews, but very few job applicants were placed.

"2. A situation where investigators in a law enforcement agency were given a quota of eight cases per month. At the end of the month investigators picked easy fast cases to meet their quota. Some more urgent, but more difficult cases were delayed or ignored.

"3. A manufacturing example similar to the above situation where a production quota caused managers to work on all the easy orders towards the end of the month, ignoring the sequence in which the orders were received.

"4. Another case involved emphasis on setting monthly production records. This caused production managers to neglect repairs and maintenance.

"5. Standard costing is mentioned as a frequent source of problems where managers are motivated to spend a considerable amount of time and energy debating about how indirect cost should be allocated and attempting to explain the differences between the actual and standard costs."

You see the general point. In each case, a metric is defined in the hopes that it will drive organizational behavior in a good direction. But the people working inside the organization naturally want to score as well as possible, preferably without too much effort. So they use their creativity to find ways to boost the numbers.

Also, in case this discussion sounds familiar, we have seen these themes before. Once was in 2021, in this post here, where I argue that "There is no metric in the world that cannot be gamed." But the exact same point shows up in this post here from 2023, about systems thinking—where the fundamental insight is that if you design your operations and metrics in a lazy way, without thinking through what you are doing, you will incentivize your people to deliver bad service. 

Pro tip: Don't do that. 

Goodhart's law

Let me wrap up by referencing the webcomic xkcd. This one is about Goodhart's Law, that "When a measure becomes a target, it ceases to be a good measure." Of course the reasons behind Goodhart's Law are everything I've already said in this post. Here's what xkcd does with it:****

Meanwhile, I hope everyone has a great holiday season! I'll be back in a week to talk about the third assumption we make regarding metrics.

__________

* It seems that this formulation is from a summary of Ridgway's work by the journalist Simon Caulkin. See this article for references.   

** Ridgway, V. F. 1956. Dysfunctional consequences of performance measurements. Administrative Science Quarterly 1(2): 240-247. See reprint available here, or summary available here. 

*** These five examples are quoted from this summary here, by James R. Martin, Ph.D., CMA.   

**** The xkcd website makes the following statement about permissions for re-use: "This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License. This means you're free to copy and share these comics (but not to sell them). More details."     

      

The hidden bias inside metrics

Last week we talked about metrics, and about how—if you find you need to measure something where no metric has ever been established before—you can just make one up. Of course this is true, but you still have to be careful. Make sure you understand what you want the metric to tell you. The reason is that sometimes you can measure the same thing in two different ways, and each way conveys a hidden message or bias.

For example, suppose you are comparing different ways to travel from one place to another: walking, skateboarding, bicycling, driving, flying. And suppose you want to know which is the safest. How do you measure that?

It all depends which one you want to win. If you work for the airline industry, then you probably want to convince people that commercial air travel is the safest form of travel. That way, more people will choose to fly, and your business will grow. So in that case, you measure safety in terms of Number of fatal accidents per mile traveled. 

It's a simple fact that commercial air travel has very few fatal accidents, so the numerator of that fraction will be very small. At the same time, flying is most practical when you want to cover long distances, so on the whole the denominator is very large. That means that the overall fraction will be very small indeed, and—sure enough!—the airline industry regularly advertises that flying is the safest way to travel.

But you could equally well approach the question from another direction. Suppose you ask: If something goes wrong, how much danger am I in? Using this metric, flying no longer leads the pack. If something goes wrong while you are walking—even if you are walking long distances—you likely need no more than a day's rest and a better pair of shoes. But if the airplane that you are on develops catastrophic engine failure at 35,000 feet, the odds are strongly against anyone walking away from the experience.

This is what I mean by the "hidden bias" in a metric. Because metrics are (by definition) objective and (generally) quantitative, we tend to assume that they are unbiased. But when you try to measure "Which form of travel is the safest?" flying comes out as either the best or the worst, depending which metric you choose.

Nor can you ask, "Well which one is the right metric to settle the question?" There is no "right" metric. Both of these metrics answer part of the question about safety. The real problem is that the question about the "safest form of travel" is badly posed. What are you really asking for? Do you want to know about the frequency or likelihood of serious problems? In that case, flying is the safest. Do you want to know about the lethality of serious problems? In that case, flying is the most dangerous. Before you choose a metric, you have to understand very exactly what you want it to tell you. In the same way, before you blindly accept any metric quoted by somebody else, think hard about what that metric is really measuring, and about why he chose to use it and not a different one.

Years ago, I saw a customer advocate on television exploding a metric in the most delightful way. Some brand of potato chips had come out with a new line, that advertised "Less Salt and Less Oil!" But a close analysis of the production process showed that actually a bag of the new chips contained—overall—more salt and more oil than a bag of their regular line. How could they get away with advertising "Less Salt and Less Oil"? When he challenged them they explained that they had made the potato chips smaller! Therefore—so they said—if you sit down with a plan to eat exactly ten potato chips (or some other definite number), you end up consuming less salt and less oil than if you had eaten ten of their regular chips. And of course the consumer advocate riposted with what's obvious, namely, that nobody ever sits down to eat a specific number of potato chips. In fact, he said, the only time he had ever seen anyone count out a specific number of potato chips was when he saw two eight-year-old boys dividing a bag between them. Otherwise, that's not what people do. So the metric was true as far as it went, but it was misleading.  

The same thing is true of any other metric. Be it never so objective, it will silently push the conversation in one direction rather than another. When you choose a metric—or when you make one up, if you have to do that—make sure that it is pointing in a direction you want to go. 

      

"How hot is that pepper?": Adventures in measurement

We all know that measurement is important. But what if you want to measure something that has no defined metric?

The answer may be that you have to make something up. Look at the feature, or process, or event that you have in mind; determine its salient characteristics; and then decide how those can be best isolated and communicated. Often, the clearest communication is quantitative, in terms of numbers. In a few cases, you might find it simpler to communicate in binary terms (on/off), or qualitatively. But in all events, make sure that your distinctions are objective and repeatable.

The basic elements you have to define are:

• system of measurement
• unit of measure
• sensor

And that's it! Once you know how you are going to check the thing (sensor) and what you are going to count (unit of measure), you can measure whatever you need.

This video explains the process by walking through the steps to establish the Scoville scale, which measures the hotness of chili peppers. It's quick and fun and less than two minutes long.

  

 

Another take on 5-Whys

A couple months ago, I saw a fun cartoon on LinkedIn. It was attached to a perfectly sound article which you can find here. But the cartoon is what I wanted to share.

And of course this is right. Sometimes analysis sputters out. You might not have the data you need to get a first-class root cause. Other factors might interfere as well. This work can be hard.

It's still valuable to do, of course.

                

How small is "small"?

Last week, I wrote about an organization I once knew that accomplished amazing results with almost no formal systems. Among other things, I said, "When the number of people is small, and when the average tenure in jobs is high, you can accomplish a lot of good work following the Everybody-Just-Knows methodology." It sounds promising.

But how small is "small"? At what size do you lose the magic of "smallness" and have to start putting systems into place?

It's sooner than you think.

A friend of mine works for a retail firm with around sixty-five employees—maybe more now and then for short-term or seasonal work. It's a "small company" in anybody's book. But a little while ago they ran into a situation that proved the need for some formal systems.

What happened is this. The company generates product information sheets for their products. These sheets are used by several departments internally, and they are also made available to customers. Some time ago, the Social Media team decided to update these information sheets to make them "easier to read": in practice this meant adding some colorful graphics and rearranging the text so there is more white space on the page. For sheets with too much text to rearrange, they just deleted anything they found boring.*

But the Sales and Support teams, who interact directly with customers to resolve problems, urgently needed all that deleted text. So they in turn just kept copies of the earlier versions, and continued to duplicate those outdated versions when they needed more. At no point did anyone escalate the issue to the responsible manager to get a ruling on how to proceed, because no manager is specifically in charge of the information sheets. They could have gone to the General Manager, but no one wanted to escalate the issue that high.

When my friend told me this story I tried to discuss it pleasantly. But inside I felt like I was looking at one of those pictures you see in magazines with the headline "How Many Mistakes Can You Find?" (1) Nobody was formally responsible for the information sheets. (2) The people updating the sheets didn't know the full list of interested parties who used them, nor (3) what those parties required. (4) The updated sheets were not sent out for review before they were approved. And (5) it was possible for people to keep and use old sheets instead of having to use the current ones. There might be more issues if I think about it for a while, but that list is a good start.

I don't blame the company. No company starts out with all these systems on Day One, and it is usually through episodes like this that they learn they need more than they've got. Also I wanted to tread a little lightly because the company wasn't a client. They'd never asked for my advice—my friend was just telling me about having a bad day at work. So I made a few suggestions for her to take back to the office, to see if she could nudge things in the right direction, and left it at that.

But how is this any different from the team I described last week? If that team could do consistently good work with no formal systems, why shouldn't we expect the same results here?

The easy answer is that there are no tricks in the world that will guarantee you do good work. Even a formal Quality Management System won't guarantee that. At most, the Quality techniques will help you avoid a lot of familiar known mistakes that otherwise recur over and over. So while I say that a small team can do good work without formal systems, that doesn't mean they will.   

Beyond that, there are two or three differences between the Repair Center I described last week and my friend's retail store. That is, there are two differences which definitely bear on this question, and a third which might.

1. The retail store is small for a company, but it's nearly ten times the size of the Repair Center.
2. The Repair Center all did the same kind of work, and they all sat together in one big room. The retail store has multiple departments that are spread out across the facility.
3. Everyone in the Repair Center was an "old-timer." Most of them had worked in the same job for over a decade. The retail store has more of a mix of age and experience levels.

The retail store's problem with information sheets arose largely because some people (the Social Media team) didn't know the needs of others (the sales and support staff). The first two points above mean that could never have happened in the Repair Center. Everyone there knew what everyone else was working on—and what they needed, and almost what they were thinking—because they all did the same task and they had worked together for so long. The last point, about seniority, might have helped too, because one of the benefits of long experience is that you've already had plenty of years in which to make foolish mistakes: after a while you should have already committed most of the relevant mistakes, and learned from them.

So when I say that a small team can do good work without formal systems, there are a lot of other conditions that have to be in place instead. The team needs to be really small, small enough that all-to-all communication is easy and immediate. They have to be knowledgeable and experienced. There are personal characteristics that they have to have, as well: conscientiousness, focus, and a host of others. It's possible, but it's not easy.

If you want to grow larger, consider starting to put formal systems in place.

If someone retires, look closely at what he used to do. Often someone will start doing basic document or system maintenance on his own time, just because he sees that it needs to be done. When he leaves, it may be time to formalize it.

And remember the other extreme, just to keep everything in perspective: that you can have sophisticated formal systems in place and still make alarming mistakes. 

__________

* That's probably an exaggeration; but in light of what came next it might as well have been true.   

     

Quality work out of Fibber McGee's closet

From time to time I've written about the question whether it's possible to achieve Quality work without having a formal Quality system. (Here is one recent post where I touch on it, for example.) In general my argument is that it is always possible, but on any large scale it becomes staggeringly difficult. From this point of view, all our Quality methods are (so to speak) just tips and tricks—gimmicks, if you will—to make it easier to get the results we want.

With this background in mind, let me tell you a story.

Once upon a time, I worked for a California startup that had just been acquired by a much-larger company headquartered in Europe. One of the new initiatives was to get us integrated into their ISO 9001 system. To kick-start that activity, Mother Company sent an Auditor out from the Old Country to do a complete review of our system.

There were a lot of mis-matches. The Auditor expected meetings to start on time, but company culture at the startup meant people usually drifted in about 10 minutes late. A lot of procedure documents were either missing or badly followed. The Auditor's opinion soured quickly, and didn't improve as the week went on.

By Dell Publications., Public Domain, Link

At one point he came to our Repair Center. This was part of the Customer Service organization, and it was just what it sounds like. Customers whose units had broken or were malfunctioning could send them in and we'd fix them. If the unit was still under warranty, the fix was free; otherwise, we'd quote a price. Also, the Repair Center sold a lot of extended warranties to customers who didn't want to haggle over a bill when they needed their unit back.

The Repair Center was housed in a space that was scarcely big enough. Cabinets were full of old equipment, with tools and supplies shoved in at odd angles. Technicians regularly had half-completed jobs teetering on the edges of their workspaces, put on hold because they had to contact the customer for clarification of a question; but meanwhile they had started work on another job. There might be more equipment under the desks. It all looked like Fibber McGee's hall closet. 

Here the Auditor drew a line. He stopped at the entrance of the Repair Center and stood stock still. He leaned in to look around, but he would not cross the threshold. I don't know if he was afraid he might dislodge something or trip over it. But he looked around, scowled, murmured quietly "This center will never meet ISO 17025," … and left.

From then on, whenever anyone asked him genially, "How's the audit going?" the Auditor talked about our Repair Center. He described as much of it as he had been able to see, and explained how many regulations it failed to meet. If anyone asked how much work he thought it would take to bring it up to snuff, he just shook his head sadly.

Sooner or later, word got to our Head of Operations. This fellow was a long-time employee of Mother Company, but he was an American; and he had been sent out shortly after Mother Company acquired us, to offer a guiding hand as we assimilated into the larger organization. So he asked for a meeting with the Auditor. I was the Auditor's official host, so I tagged along.

Head of Operations: I hear that you don't like our Repair Center. What's wrong with it?

Auditor: It's cluttered. It's disorganized. There is no control over the space or the tools or the work. I don't see how they can function.

Operations: Compared to what? Are you comparing it to the Customer Equipment Laboratory in the home office, back in the Old Country?

Auditor: Yes, exactly.

Operations: Well help me understand this. Because the Repair Center here is a profit center, that generates something like 25% of the company's annual revenue. Meanwhile that Laboratory you are comparing it to is a cost center, and every year those costs are significant.

Auditor: I'm not concerned with how the company organizes its departments financially. I'm looking at compliance and customer satisfaction.

Operations: Fine, but that's another point. Back on my desk I have a stack of customer complaints an inch thick, complaining about the Customer Equipment Laboratory in the Old Country. And in my files I have just as many notes from customers expressing gratitude for the quick and professional service they got from this Repair Center. So who's offering more customer satisfaction?

Auditor: ISO 17025 has clear requirements that laboratories have to meet, and this Repair Center doesn't even come close.

Operations: Does that standard apply to this Repair Center?

Auditor: I don't know, but I am going to check.

In the end, it all worked out. The Repair Center tidied up their space, labeled their cabinets, and adopted some rules about how to handle work if you have to interrupt it to wait for a customer callback. After some research, the Auditor determined that ISO 17025 did not apply to the Repair Center's work, and the tidying that they did was good enough for ISO 9001.

But how were they able to do such good work all along? Isn't tidiness important? Does it really not matter whether your storage cabinets are labeled?

Of course it matters. The key is that everyone who worked in the Repair Center had been there for a decade!* This was a small company, with a dedicated staff and very little turnover. They didn't bother to label the cabinets, because everybody just knew where everything was. They didn't write down a lot of their operational procedures because everybody just knew what to do. They didn't bother to label work-in-progress because everybody just knew who was working on what.

And this is a valid Quality strategy for some organizations. When the number of people is small, and when the average tenure in jobs is high, you can accomplish a lot of good work following the Everybody-Just-Knows methodology. The only problem is that this approach doesn't scale well, and it's hard to bring on anyone new when your employees start to retire or die. 

As soon as you start to grow, or as soon as you have to work with anyone new, the traditional Quality tools suddenly become a lot more useful. 

__________

* Strictly speaking I think the New Guy had been there only seven or eight years.   

        

Who should be in charge?

Here in the United States, we've just finished holding an election; so now we're taking down the signs, sweeping up the confetti, and getting back to normal. I've been browsing the Internet, reading posts where people on one side of the aisle congratulate each other, while those on the other side console each other. (As a Quality nerd, I naturally wanted to write-in W. Edwards Deming for President, but unfortunately he died in 1993.) But I also started to wonder, Does the Quality profession have anything to tell us about who should lead an organization? About who should be in charge? 

In the most immediate sense of the question, No, it doesn't. The ISO 9001 standard contains requirements for what top management shall do, and its companion volume ISO 9000 contains discussion about the concept of leadership. But there are no ISO requirements anywhere that say "Name George as CEO, and not Fred." On the other hand, if you take a step back to look at management theory more generally, it's possible to find some indications.

One of the most remarkable sources of guidance that I've found (on a couple of topics, but this is one of them) is a short essay by Peter Drucker called "Managing Oneself." Drucker's basic point in the essay is that, in order to be effective, you have to know how you work. Then organize your work along those lines, so that you can produce results. And he gives historical examples of leaders or other executives who failed to heed this advice, and who thereby undercut their own achievements.

Drucker approaches this topic from a lot of different angles, and I strongly recommend that you read the article regardless what your current work happens to be. But part of his advice on understanding how you work can be adapted into advice on who should occupy certain roles. (In what follows I have modified Drucker's terminology a bit, to make my own point clearer. But I promise he says these things. 😃)  

Specifically, Drucker points out that there are people good at thinking, and there are also people good at deciding—but they are rarely the same people. People good at thinking thrive as advisors, or technical specialists. They can analyze a situation thoroughly, and map out the three approaches most likely to succeed. But when you ask them to pick one, they see too many subtleties and hidden implications, and so they get lost trying to play out multiple chess-matches in their heads. Wait, if I move here then he'll move there, and that means I have to ….

By contrast, people good at deciding may not be able to understand all the ramifications of this or that policy unaided. They definitely need the support of their advisors or technical specialists to clarify, If you choose X, then Y is sure to happen as a result. But once they have all the facts, they know exactly what course they want to take, and they are happy to bear the responsibility for choosing it. 

With this distinction in mind, Drucker is absolutely clear that "The top spot requires a decision-maker." On the surface this may sound obvious, but notice the corollary: the top spot does not require a brilliant analyst. The CEO does not have to be the smartest person in the room, and in most cases shouldn't be. Normally, it will be better if the smartest person in the room is an advisor that the CEO can trust, who maps out the options and highlights the pitfalls of each. Then the CEO picks one, and the organization moves forward.

Oliver Wendell Holmes, Jr. once characterized Franklin Roosevelt as having "a second-rate intellect but a first-rate temperament." Most of the time, that's the right mix. 

     

Quality and Despair

It's Hallowe'en. Once again, it's time for something spooky.

This year I'm going to quote a commercial product. It's a Demotivational® poster from the folks over at Despair Inc. It starts with one of the basic commonplaces of the Quality business, and spells out the awful consequences of taking it literally.

The race for quality has no finish line;

so technically, it's more like a death march.   

Click here to jump to their site, where you can order these words of wisdom on a poster or a mug.

Happy Hallowe'en, everyone!

    

Outsourcing your evil deeds

Last week I wrote about the peculiar fact that multinational corporations seem able to avoid unwelcome regulation, at least in certain cases, by the expedient of quitting the country to go elsewhere. This approach is admittedly a bit extreme, so today I want to talk briefly about another—much more common—way that some companies try to get around rules that forbid them to do Bad Things: hire a scoundrel as an external supplier to do the Bad Thing for you. That way your hands are clean … well, more or less … and dealing with the regulators becomes his problem.

This approach has become so common in international trade that it almost passes without comment. If your company makes widgets, and if you are regularly undersold by competitors because wages in your country are so high or environmental regulations add extra costs, someone is sure to suggest that you relocate your factories to another country where neither of these considerations is in play. Alternatively you can outsource the actual manufacturing to a supplier in the other country. That way you aren't paying the low wages, and you aren't causing the adverse environmental effects; but you still get your widgets a lot cheaper than before.

Of course, any such gains are temporary. If you can save money by moving your manufacturing to Ruritania, so can your competitors. Then some other country comes into view, with even lower wages and even worse environmental protections, and everyone moves there instead. Soon countries are competing against each other in a "race to the bottom." As I have discussed in an earlier post, in the long run nobody wins such a race. But in the short run, some companies find it compelling; and after all, "In the long run we are all dead."*  

I assume that the same dynamic probably operates domestically as well. That is to say, I have no personal knowledge of any domestic companies who exist so that their clients can skirt inconvenient legal or ethical constraints, but it wouldn't surprise me. Some people don't mind sketchy work, and some people will do anything for a price. 

The good news is that ISO 9001 explicitly disallows this! Clause 8.4.2(a) of ISO 9001:2015 states clearly that:

The organization shall ensure that externally provided processes remain within the control of its quality management system; …. 

So if you design and sell a product, but you outsource its manufacture to someone else, you are still responsible for what they do.

Most of the time, this responsibility is for very practical reasons. Maybe you do much of your own manufacturing, but there's one specific process that you outsource. Well if you require that all your manufacturing equipment must be calibrated to a specific tolerance, don't you want to flow down that same requirement to the supplier who is executing this one special process? If you don't, their uncalibrated equipment might ruin all the exactitude you achieved with your carefully-calibrated equipment, and you'll have to scrap the whole lot. Nine times out of ten, or 99 times out of 100, this is the kind of "control" that really matters. Mostly ISO isn't afraid that you are going to try to do Bad Things in an underhanded way, because most people just don't do that. But ISO is concerned that when your process is executed, you get what you want.

To be clear, this clause does not mean that you have to know your supplier's business better than they do. It does not mean that you have to define the details of their operating procedures. The whole reason you are hiring them should be that they are experts in whatever you want them to do. 

Nor does it mean that if your company has to be certified to AS9100 because you are building aerospace parts, then the caterer that you hire to provide lunch has to be certified to AS9100 as well. (I hope that's obvious.)

But if you have any overall constraints that apply to all of the work inside your QMS—like the calibration example I just gave—then (where it is relevant and meaningful) you have to flow down those requirements to your supplier.

And if you happen to be the one case in 1000 who wants to get away with a Bad Thing by hiring a scoundrel to do it for you, … don't. Just don't.  

__________

* John Maynard Keynes, A Tract on Monetary Reform, 1923. Quoted many places around the Internet, for example here.       

     

When ISO 9001 Fails

This morning, Quality Digest published my article, "When ISO 9001 Fails." It's their article now so I won't post the text of it here, but you can find it by following the link. I hope you find it useful!

            

The Braganza gambit

And now, if I may digress momentarily from the main stream of this evening's symposium,* … I'd like to raise a question which relates more to regulatory compliance than to Quality per se, but which has bothered me from time to time, and which seems to lurk on the margins of other—more normal—Quality topics. (In fact I plan to discuss one of these next week, in a follow-on essay.)

The background is this: First, we all know there is such a thing as global trade. In fact, the whole point of international standards is to facilitate global trade. As I explained once in this forum a couple of years ago, "A standard is like a common language: it allows us to do business with strangers, because we know that we are both talking about the same thing."

Second, we all know there are global (or at any rate multinational) corporations. Over the course of my career I've worked for at least two companies headquartered in Europe (LM Ericsson and Robert Bosch), even though both times my local office was in southern California.

Third, we all understand more or less how companies are regulated. Some authority codifies a set of rules: those rules might be voluntary (like ISO 9001) or legally mandatory (like health and safety regulations). Then the company decides whether they want to abide by these rules. (In the case of mandatory legislation, we should assume that the answer is always Yes.) If yes, the company takes steps to implement the rules; and if they fail, there is some kind of system in place whereby someone can complain. When the authorities get a complaint they check the facts; and if the company has indeed failed to meet the requirements, the authorities react accordingly. In the case of ISO 9001, the responsible Certification Body can decertify the company; in the case of legal noncompliance, the relevant government can impose civil or criminal penalties.

Now finally here's the question: How do you regulate an international company?

I fear that the answer may be: Mostly you can't. I'll explain why, but I would be delighted if you can show me where I am wrong.

Let's say that some local company violates a local regulation. Government inspectors come out to check the status, and—depending on the severity of the issue—they might give the company written notice to correct the problem in a defined time, or they might padlock the doors. If company personnel try to interfere with the government inspectors, they can be arrested. And since it's a local company, that's all it takes to stop them doing whatever Bad Thing they were doing. Problem solved.

Suppose that the company has multiple branches in the same state: then, depending on the nature of the Bad Thing that Law Enforcement is trying to stop, they might have to take a heavier approach. Or they might leave the branches alone but target headquarters. If the company has branches all over the United States, Law Enforcement has to get more ambitious still, because sometimes state laws disagree (so the Bad Thing might be legal in another state). Also, local Law Enforcement is unlikely to have jurisdiction in another state, and so will have to coordinate with other agencies in order to stop the Bad Thing once and for all.

But if the company has offices all over the world, then what? The very most that American Law Enforcement authorities can possibly do is to arrest whichever company personnel happen to be located inside the United States. But they are powerless over the offices in Ruritania or Grand Fenwick.    

In the ordinary course of things, a multinational company will probably find it convenient to comply with routine local regulations, because they will see those regulations as just a cost of doing business. As long as the opportunities in a country are bigger than the costs, they are likely to cooperate. But this cooperation is strictly a voluntary choice on their part. In an extreme case, they can always shut down the local offices and leave.

This strategic departure from a country because you don't like the laws is what I call the Braganza gambit. The Braganza family ruled Portugal and the Portuguese Empire from 1640 until 1910. During the Napoleonic Wars in the early nineteenth century, Napoleon Bonaparte installed many of his relatives in thrones across Europe. His method—used for example in the Peninsular War against Spain—was to defeat a country and capture the royal family; then he could force them to abdicate in favor of one of his relatives and move on to the next country.

By Lumastan - Own work, CC BY-SA 3.0, Link

But not in Portugal. The Braganzas saw what Napoleon was doing and realized they were next. So they moved the entire royal court to Brazil, which was at that point part of the Portuguese Empire. When Napoleon conquered Lisbon, the royal family was nowhere to be found. (In the end they liked it in Brazil, and didn't move back until 1821—long after Napoleon was no longer a threat.)

So there you have it. Multinational corporations have the privilege—unavailable to local corporations—that they can (within limits) decide which legal regulations they feel like following. And in case any regulation is too burdensome for them to tolerate it, they have the option of leaving the country.** 

If a multinational corporation decides to use the Braganza gambit to avoid an onerous regulation, about the only leverage the abandoned country has is to close its markets. "If you won't abide by our rules, you can't sell your goods here." Whether that's a meaningful threat depends very much on the particular details, and of course sometimes the same maneuver plays out in reverse: a company might refuse to sell into a certain country until this or that policy is changed. It is hard to generalize about how effective either tactic is.*** 

What do you think? Am I wrong? Is there something I've neglected?

Or can multinational corporations escape troublesome regulation just by moving abroad? 

Please leave me a comment with your perspective. 

__________

* Tom Lehrer, introduction to "The Elements," Reprise/Warner Bros. Records, track 4 on An Evening Wasted With Tom Lehrer, 1959, LP record.  

** There's even a related line of thought that protects international organizations. Concretely, if you or I (as private citizens) feel wronged by some decision from an international organization like the WEF or the ISO, we may find it hard to sue them for redress because it's not at all clear which court—if any—has the appropriate jurisdiction.

*** Certainly this is the logic behind international economic sanctions, where—in this case—one government requires all the companies subject to it to avoid business in another country until that other country changes its policies. When small countries are subjected to coordinated sanctions, the effects can be crippling. When large countries are subjected to them, the results are not so immediate. Consider, for example, this recent video by a YouTube creator "Eli from Russia," who publishes travel information (and strictly avoids politics). She describes the impact of sanctions on Russia, and the results have been (to say the least) not uniform.

    

What's "proportionate"?

When the ISO 9001 standard requires you to take action to address your risks and opportunities, it includes this admonition in clause 6.1.2: Actions taken to address risks and opportunities shall be proportionate to the potential impact on the conformity of products and services.

OK, I guess that's fine, but what does it mean? What does it take for your actions to be "proportionate" to the risks they address?

The word is never defined—or at any rate, neither ISO 9001 nor ISO 9000 define it. But intuitively I think we all have a sense for what it means, don't we? The basic idea is ancient: Nothing in excess.* If you face a risk that might, at worst, cost you $100, then it is foolish to spend $1000 to prevent it. That cost, or that level of effort, is disproportionate to the $100 downside that you face from the unmitigated risk. Most of the time, we probably don't need a definition more exact than that.

But "sometimes the clearest way to explain what a rangatang [sic] is, is to 'tell what it ain't.'"** A couple of months ago, I stumbled upon a blog post from 2016 that explains the concept of disproportionality with crystal clarity. The author of the post—Quinn Dunki of Blondihacks—just wanted to set up an automated cat feeder, so that her cat would be fed on time and she didn't have to watch the clock every day. Simple, right?

Turns out her cat had a different idea. Her cat's idea was, "How do I get this machine to give me more food than Quinn wants me to have?" So Quinn had to make some adaptations to her automated cat feeder, to protect it from the prying paws of her cat. As she says at the opening of her blog post, "The trick is to be smarter than the animal with a brain the size of a walnut."

But of course, Quinn worked on this problem part-time, and her cat worked on it full-time. 

You can read the results here.

In the end, Quinn won. But I'm pretty sure nobody would say that the effort she expended was proportional either to the benefits she gained or to the risks she was avoiding.

Verbum sapienti sat. 

__________

* "Μηδὲν ἄγαν" was one of the three proverbs said to have been inscribed at the entrance to the Greek temple at Delphi. See here for more information.  

** Owen Ulph, The Fiddleback: Lore of the Line Camp (San Francisco: Browntrout Publishers, 1995), p. 23.      

How do you prove "consideration"?

How many times have you seen departments do things that don't help them any, just because it's easier for the auditor once a year? I've seen it too often to count, and it's never the right thing to do. Oh sure, in a sense I appreciate it when I'm the auditor. But also, it's really unnecessary. I've audited a lot of departments over the years, and they've done things a lot of different ways. If it works better for you the other 364 days of the year to do this rather than that, … well, as long as it meets the rules I can probably figure it out.

I was thinking about this recently while talking to someone about the rules for management review. Right now, ISO 9001:2015, clause 9.3.2 states, "The management review shall be planned and carried out taking into consideration"—and then there follows a long list of topics, (a) through (f), where item (c) is further divided into seven subtopics. It's a comprehensive list. Anyway, my friend was saying he wishes the ISO would change this requirement to say that management review must explicitly include all these topics, because "How are you supposed to prove consideration to an auditor?" What he meant, of course, was that if the agenda for management review were required to include every one of these topics and subtopics, it would be easy to show that you had "considered" them all.

Long-time readers may remember that I think this is a terrible idea! The biggest risk in any management review is that the participants are likely to get bored. To avoid boring them, cut out everything you can. Discuss only the pain points that have to be resolved by the specific participants of this meeting. That means that if your internal audits or your supplier evaluations are all green, it's enough to wave your hand and say so; you don't have to drag the attendees through an itemized list of each one. Spend your time instead explaining that it's time to buy a new widget-stamping machine, because the old one slides out of alignment once a month like clockwork and the rework costs are eating you alive. 

But of course you still have to pass that audit once a year, so how are you going to do it? It's all very well for me to say that you shouldn't rearrange your whole management review just for the convenience of the auditor, but you are going to have to show some kind of objective evidence. What will it be?

Do it like this.

First, as you prepare the meeting, go through every single one of those topics listed in clause 9.3.2, and document where it stands right now. (You have to do that anyway in order to find out where your pain points are, since those are the topics you will discuss.)  

Second, while you are conducting the meeting, keep all this material handy where you can reach it. Maybe this means it's stored electronically just one click away, or maybe it's on paper in a notebook on the desk next to you. But just in case someone brings up a question about one of those topics you thought you could afford to skip, make sure the data is immediately available.

Third, store all this data as a permanent Quality record, together with the minutes from the management review meeting that it supported.

Fourth, ask your internal auditors to look for this data when they audit the management review process, just to keep you honest. 😃 Naturally whoever audits the Quality function doesn't work for you—do they??—so if you make a mistake they won't be shy about writing it up.

And finally Fifth, when the external auditor visits, pull out all this stored data as proof that you really did consider all the topics listed in the standard. Then you can explain why you tailored the agenda to address the problems that really needed management attention, and why you skipped over all the topics that were functioning smoothly because they were just business as usual. 

Simple. Straightforward. And you don't need to "include" all those topics in the review in order to "consider" them. 

      

"Lazy compliance" and unintended consequences

Last week, a friend of mine got a new electric stove. But that was only the beginning of the story.

Once it was installed, she learned that her new stove wasn't compatible with her cookware. I didn't know that was possible, but she explained it to me as follows: 

Turns out all electric stoves manufactured since 2018 need to meet a safety standard to reduce fire risk which cycles the burner off when a pan is not in contact with it. 

In complete contact with it; to work on my "sensi-temp" burners, pans have to have a completely flat bottom.

My only 2-quart saucepan was warped.

So, this morning I purchased a new one.  Stainless steel, $28, sigh.

But at least I can cook rice tonight!

This is the kind of outcome that makes people believe conspiracy theories. To her credit, my friend didn't start yelling that the stove-manufacturers must be in cahoots with the cookware-manufacturers to drive up sales—at least, she didn't say it around me—but I would have understood if she had. For myself, I began to wonder how such a defective outcome came to pass in the first place. 

Should I take a minute to explain what it is that makes this update to new stoves an impairment and not an improvement?

It's unexpected. Maybe there was coordination inside the kitchen-appliance industry, but I don't remember seeing any communication to the general public back in 2018 that stoves were changing in a fundamental way. Nor does my friend, obviously.

It requires additional actions from consumers, unrelated to the stove itself. When the auto industry introduced anti-lock brakes, they didn't insist that drivers start braking differently. Rather, they started from the known facts about how drivers react in panic situations, and redesigned the brakes to fit the drivers. By contrast, telling home-cookers that they have to buy new cookware to fit their new stoves is completely backwards. 

It requires additional and unexpected costs. This is basically the same point, hitting your wallet as well as your habits.

It's going to break. Every feature that you add to a product is one more thing that can break. But what happens if the sensor breaks on this new stove, while the shutoff-mechanism for heating elements remains intact? Then the stove will fail to detect a pan on the burner even when a pan is there, and the burner won't heat up. Result: the stove won't work. Any change that makes a product more fragile and less resilient is an impairment and not an improvement.

Sure enough, people hate it. There are long discussion threads on Reddit (see here, for example) about how to replace your brand-new "sensi-temp" burners with the old style, where to buy old-style burners as replacement parts, and so forth. Of course, buying old-style burners from some random site on the Internet means you have to spend even more, but plainly some people think it is worth the expense. 

Ironically, I bet that dismantling your stove to replace the heating elements with ones you ordered online may introduce some safety risks into your day; and yet I guarantee that when GE and other manufacturers did a safety analysis on this change before implementing it, they never considered the risks from home retrofits by angry customers. But this change to stoves has created a market and a community for exactly this kind of home retrofit. 

Why did they do it?

I wish I knew. I would love to see the FMEA carried out on this change before it was implemented.

It would be easy to blame ignorance. Maybe the engineers never actually do their own cooking, and didn't realize that this new feature could pose a problem. But that can't be the whole story, because GE (at least) provides an information sheet that warns explicitly about non-compliant cookware. You can download this sheet from the GE website here. (Also I have archived a copy locally here, in case the GE site is ever rearranged.) 

At the most basic level, the change appears to have been introduced in response to an update of the safety standard UL 858, "Household Electric Ranges." [You can buy the standard here.] Based on other informational material provided by GE, the thinking appears to have run something like this:

• Stovetop oil and grease fires are bad.
• To prevent them, we have to prevent pans on the stovetop from getting too hot.
• To measure how hot a pan is, we have to have a sensor touching the pan.
• But this approach could fail if the sensor doesn't touch the pan. So to avoid that case, if the sensor doesn't touch the pan we will cut off power to the burner. (According to this article, the stove should not cut off power completely; but my friend's experience was exactly that.)

It's logical, as far as it goes. But the whole line of reasoning exemplifies what another friend has called "lazy compliance": that's "where they make changes required for safety without bothering to make compensatory changes so the thing works as well as it did before."

Of course, you might think, So what? It's safer, and that's what counts. And to some extent, naturally that's true. But in the rest of our lives, we are often willing to make trade-offs where we sacrifice a measure of safety to secure nothing grander than convenience. 

For example, if all automobiles had to observe a strict maximum speed limit of 20 miles per hour, the number of fatal car crashes in a year would drop to nearly zero. Shall we take a vote? Never mind, I guarantee the measure would fail. Even though "car crashes are the leading cause of death in the United States for people ages 1 to 54," we'll never get a voting majority willing to eliminate those deaths by lowering the speed limit so far.

So why stovetops? And why didn't those of us who use stoves regularly get a vote? (If you remember last week's post, that's called "stakeholder engagement," and it's important.)  

I wish I knew. If you know more about it than I do, please leave a note to help explain.

     

Know your stakeholders

Change can be hard. This is true for many reasons; but at an organizational level, one of the main reasons is that the people involved in or affected by a change may resist it. Maybe the change will be too hard for them to learn, maybe it deprives them of benefits they were getting under the old system, or maybe … well, they could have all sorts of reasons. But if a few of them dig in their heels, the whole job of implementing even a minor change becomes exponentially harder.

Of course this is why change management is a field, and why stakeholder engagement is one of the main topics in that field. Over the years, practitioners have discovered that some methods work better than others to win people over to your side when you are trying to implement a change. Today there are organizations, like the Change Management Institute, that focus on nothing else.

Yesterday I had the opportunity to attend a webinar sponsored by the Quality Management Division of ASQ, which introduced the CMI's Body of Knowledge (CMBoK™). The webinar was presented by Douglas Wood and Sandy Furterer, both of ASQ. Of course, in a single hour the presenters could touch on only a very few topics, but I found it useful all the same. And for those already looking ahead to next year, Wood and Furterer promised they will be back in 2025 to present an overall change management approach for middle managers. Stay tuned! (Yesterday's webinar was part of a series that began last month with "The Faces of Change Management.")

I want to talk today about a tool that Furterer described for evaluating your stakeholders and planning how to engage with them. Conceptually it is very simple; but that simplicity makes it useful and powerful, because it is so straightforward to apply.

The tool is just a table that lists all the stakeholders involved in a particular change initiative. It has six columns, as follows:

Stakeholder: List each stakeholder by role, not by name. Use the structure of the subject process to identify every single role that is affected in any way. List them all. 

Type: A role that is directly affected or impacted by the change—one that actually touches the process or system or technology—is primary. All others are secondary.

Primary role: What does this stakeholder do in this process? Why are they listed here?

Potential Impacts/Concerns: What does this stakeholder care about? What are their fears, or concerns, or issues? What matters to them? Note that sometimes when you do this analysis, you'll find that one of your roles has several unrelated concerns. Maybe you identified "Customers" as a stakeholder, but it turns out that domestic customers have very different concerns from customers abroad. If that happens, it's a sign that you need two rows. You have just learned that Domestic Customers and Foreign Customers are actually two different categories of stakeholder.

Initial Receptivity: What does this stakeholder think about this change today? Where are they now? Are they for it or against it? Furterer's example included three ratings: Strongly support, Moderately support, and Neutral. (I suppose that for completeness your table should also allow for Strongly opposed and Moderately opposed, although naturally we'd all like to avoid that case if possible.)    

Future Receptivity: Here is the critical planning question: Where do you need this stakeholder to be (with respect to this change), before your plans can go forward? Note that you don't necessarily need strong support from every stakeholder. It all depends on the details. Of course you hope to avoid strong opposition from anyone; but for some stakeholders it might be enough if they are neutral, so long as they do not actively interfere. For others, if you do not have their strong support you will get nowhere. So work out what target state you need to achieve in the mind of each stakeholder. 

Now look at the differences between your last two columns: Initial Receptivity and Future Receptivity. Those differences tell you where you have work to do, before you can launch your change initiative in earnest. Identify which stakeholders you have to win over, and how far you have to win them over. Then look at what they care about, in order to figure out how best to approach them.

For example, if one stakeholder is currently opposed to the change and all you need is for them to be neutral, you can focus on addressing their fears. Find out why they oppose the change, and then show them how those concerns have already been considered in the planning. You might not get their active enthusiasm; but if you can show them that their fears are ungrounded then you may move them from opposition to neutrality.*

If another stakeholder is neutral—or even opposed—and you need their support, you have a different job. This time you have to show them how they will positively benefit from the change you have in mind. Answer for them the basic question, "What's in it for me?" Presumably you already have a good answer: you wouldn't be doing this change if it weren't on balance a benefit for everyone. But make sure that your stakeholders understand what it is before you ask for their support.

It's a simple tool. But it shows you at a glance what your stakeholder strategy has to be.  

__________

* I assume that their concerns really have been addressed in the planning already! If you've made these plans without even thinking about their concerns, maybe it's time to go back to the drawing board. 😀