"There's an app for that!"

Once upon a time, if you called a company because you had a problem with an order, you were transferred to a customer service representative who resolved the problem. But more and more often these days, it seems that you are given an app which lets you resolve the problem yourself. Often, these apps are advertised as a way to bypass the company's bureaucracy, and therefore as a customer benefit. One hears of travelers whose flights have been canceled, and who use the apps to rebook new flights while everyone else is still standing in line at the complaints desk.

So I want to ask: Do these apps represent an advance in product Quality?

They are certainly sold that way. But I think the final answer is, No. No they don't.

Even on its own terms, there is something ambiguous about the benefit provided by an app. On the one hand, yes it is absolutely true that a company's app lets you do things for yourself rather than requiring you to persuade someone else to do them for you. But notice that this power costs you something. Before you can "do for yourself," you have to know what to do, and you have to understand how to do it in the app. 

Solving your own problems

When you talk to a human customer service agent, all you have to understand is your own problem. Then the service agent translates your problem into terms that the company's system understands, and offers you a couple of choices. If you don't understand what those choices are going to mean for you personally, the agent explains. Finally you pick the option that looks least bad, the agent types your choice into the system, and you can go on about your business.

With an app, there is no one to translate and no one to explain. You have to understand not only what you want, but what the company is offering; and you have to be able to shift back and forth easily between the two perspectives. I think this is why apps are so frequently described in terms of air travel. We all know that there are a few people in the world who travel a lot—far more than the rest of us. These people understand the ins and outs of the system, and are ready to benefit by their familiarity; but the whole task is made easier for them because there are so few options on offer. If you find yourself unexpectedly stranded at the Samarkand International Airport, or at the North Pole, pretty much your only option is to rebook on a different flight. Wherever you were originally trying to go, you're not going to change your mind suddenly and decide to stay for a few months in Samarkand [it might be a nice place, but you likely don't have the right visa] or at the North Pole [because it is very cold]. So it is comparatively easy to switch from thinking like a customer to thinking like an airline.

The second job you have to master is to understand how the app itself works. Some are easier, and some are more difficult. Again, with a human agent, you don't have to take on this task. The human agent works with the same system all day, every day, and knows exactly what to do. Confronted with a brand-new app, you don't have that advantage.   

Why do we have apps?

But there is another side to this question. Why do we have customer service apps in the first place? Why are companies investing in them?

It's not because they are trying to be nice.

The motivation, surely, is that customer service apps cost companies less than customer service departments. Whizzbang Corporation can release an app, lay off the corresponding department, and be money ahead.

From the customer's perspective, for the reasons I sketch out above, replacing a human agent with an app often results in reduced levels of service. In this respect, the advent of customer service apps is just one more aspect of the slow tide of product enshittification that I described a few months ago. But in this one case, the introduction of a customer service app is a form of product degradation that can be sold as customer empowerment! There aren't many of those, so it is no surprise that companies are jumping on the bandwagon of this particular one.

What's behind it?

Why do we see such a trend towards product enshittification? Opinions differ. Cory Doctorow, who invented the word, blames it on rent-seeking by the providers of new online platforms. Other people have taken the concept and run with it. I browsed the Internet for references before writing this post, and found an article by someone who insists that global elites are consciously trying to destroy the world. (Maybe I read the piece too quickly and missed a little nuance somewhere, but consider yourself warned.)

For myself, I'd actually rather extend a little charity to the companies caught up in this cycle, even as I deplore the deliberate degradation of their products. John Michael Greer has argued that we are entering a period of prolonged economic contraction; and he has added more recently that enshittification (which he calls "crapification") is one of the normal strategies by which companies caught in an economic contraction try to keep their heads above water. Their costs are going up, and their revenues (measured in real dollars) are going down. Either they go out of business, or they have to cut the budget somewhere. So hamburgers get smaller, houses get flimsier, appliances break in five years instead of fifty, and customer service departments are replaced by apps. It's sad, but it's a much bigger problem than the avarice of a handful of tech executives. 

At the same time, even if the companies had no other choice, rolling out these apps isn't really Quality.

           

Quality out in the open

Over the last couple of weeks I've written about a recent disagreement between me and the organizational hierarchy of ASQ (American Society for Quality), and about what that disagreement can tell us about organizational behavior generally. (See specifically this post and then this one.) The situation has not really improved—not yet, at any rate—and I'll get to that in a minute. But I want to stop first to talk about an important principle that this whole discussion has brought to light.

The principle is transparency. As much as possible, manage your business out in the open. When it comes to Quality in your products or services, manage that out in the open too. (Incidentally, I make this very point with respect to process management in an article I posted just five years ago.)

Obviously this doesn't mean telling your competitors all your trade secrets. It doesn't mean exposing any information that you are legally required to keep confidential. 

But transparency generates trust. If you are open and honest with your employees, or your customers, or the public, then they will be inclined to trust you because they can see that you aren't double-dealing.

By the same token, if you are sly or silent or secretive—if you hide things—you encourage people to distrust you. They will all think, What are you doing that is so bad that you have to hide it from me? And their imaginations will conjure up pictures far worse than anything you are really doing! So you'll do better to own up to whatever it is, rather than let them imagine something horrible.

Transparency and justice

I spent my high school years at the Webb School of California. Like all private schools, it had rules. The students were teenagers, so perhaps it is not surprising that some of them challenged those rules. Once in a while, a student was expelled. 

Stephen and Ann Longley
in the 1970s.

Ongoing disciplinary actions were always kept confidential, of course. But as soon as the boy had left campus, the Headmaster (Stephen Longley, back in those days) addressed the school. Keeping the story simple and trans­parent, he told us what the student had done, and that he had been asked to leave. He was clear and to the point, with no needless detail to muddy the picture.

The impact of this honesty and clarity was enormous. In the first place we all knew that we wouldn't be seeing Fred or Herman around campus any more, and we all knew why. But there was more than that. We knew that Fred or Herman had been removed for violating an objective rule, not because someone took a subjective dislike to him. We understood that if we did the same thing he had done, we would be expelled the same way he was. But we also understood that if we didn't break the rules, we were safe. The objectivity of the rules secured us. And the relentless clarity of the communication promised justice.   

Failure to communicate

Does that mean we should just let it all hang out? Not really, because that is rarely an effective way to communicate. (Note that Stephen Longley's announcements were very focused.)

This is the mistake I made in my most recent exchange with ASQ. In last week's article, I told you that some administrator deleted one of my posts from the myASQ discussion forum on August 6, and then it took them another six days to send me an email explaining why. By the time I got that email (August 12), I was so full-to-bursting with things I wanted to say that I dumped all of it into my reply the same day. But that just meant that my email back to the administrators was full of sound and fury. Perhaps it is not surprising that they set my account to a status where everything I post is automatically queued for moderation. (Most people can post directly with no delay.) I don't know if this status will expire one day, but it was still in place yesterday (August 19). 

What I meant to say ....

Naturally I don't like being on permanent-moderation status, but I recognize that it's partly my own fault for not communicating with ASQ more clearly. But what I meant to tell them was what I am saying here and now, about the virtue of transparency.

The whole disagreement started back when I wrote about some minor mistakes in the roll-out of an updated election procedure. Very bland stuff. Someone in the administration might have felt attacked by the criticism, and in any event asked that it be deleted. The person who wrote me suggested that I take up the issue privately inside the administration.

What I wanted to explain is this:

• If my login doesn't work, that's a personal problem.
• But if a general procedure doesn't work, it already affects everybody.
• That means that people are already talking about it.
• Once people are already talking about it—and they are, today—the only way to silence them is to make the whole thing very public. If you try to suppress the discussion, people will imagine horrible things, far worse than reality.

I failed to make these points because I buried them under a lot of useless bombastic noise. But I am convinced that they are true.

Robert Townsend on secrecy

One of the best management books I have ever read is Robert Townsend's Up the Organization. Find a copy and read it. Here's what he says about secrecy.

Secrecy is totally bad. It defeats the crusade for justice, which doesn't flourish in the dark.

Did you ever ask yourself why there's a private payroll? Or why all wages and salaries aren't posted on the bulletin board? According to the lore of the free-enterprise system money is really a scorecard. So why aren't the scores posted?*

Of course the company would have a revulsion (or a revolution) if everybody had to look squarely at a list showing the salaries of the president and his nephew, who are paid four times what they're worth, and the salaries of Izzy, Derek, and Susie, who are making a third of what they're worth.

In the case of most marketing or new product planning, secrecy is sinister. It defeats your loyal opposition and protects you from your best friends when you need them most. Secrecy implies either:

1. What I'm doing is so horrible I don't dare tell you.
   or
2. I don't trust you (any more).

__________

* I'm not suggesting that you should post salaries. For one thing, it would over-emphasize the importance of money. But you shouldn't tolerate a situation in which you're ashamed to post them. Like you are. 

           

Can you report bad news?

Quality professionals have to report bad news. It's part of the job.

That's why many organizations make Quality report directly to the CEO or General Manager, instead of tucking the department under Operations or Engineering. It's not that Quality's headcount or budget are so large that they have to be managed at that level. The idea is that Quality has to report the true state of things, without the risk that someone will hide the news.

"The widgets coming off Line 2 are defective, and if we ship them we risk a $2-million recall in the next six months."

"The prototype Mark IV Fantabulator still fails five of its fouteen critical tests and leaks radiation all over the user."

"The packaging machine in Logistics is out of alignment again, even though they just fixed it last week."

These are things the organization needs to know, even though they might embarrass someone. And it is Quality's job to speak up.

Or is it?

Last week I wrote about some hiccups recently as ASQ rolled out an updated procedure for electing officers in the geographical Sections. On the whole it wasn't a dramatic topic. Then after I posted the article here, I wrote a notification to post in myASQ, which is ASQ's in-house social media platform. I started with a brief and—I hoped—non-threatening introduction:

Was anybody else as confused as I was by the roll-out of the new procedure for electing Section officers? We heard that there were changes coming, but my Section (at least) didn't get specific information until we had already missed some deadlines. I don't know if anyone else had the same experience. But is there a chance that the organization could do a Lessons Learned analysis on this roll-out, in order to make future ones go better?

I talk about it in a little more detail here:....

Then I gave a link to last week's article. In other words, I tried to keep it as bland and professional as I could, in the hopes of starting some kind of discussion about what went on.

Within fifteen minutes, an administrator deleted my post.

I post a lot on myASQ, and this is the second time I've had a post deleted. The first time was just about a year ago, when I posted an article critical of the Board of Directors. That time, the removed post was a link to this article here; and I discussed its removal very briefly in this one here. In a nutshell, last year's post was taken down ostensibly for violating the myASQ Terms of Service; but in fact it didn't violate the Terms of Service that were current when I posted it. Those Terms were updated 24 hours later, specifically to forbid any discussion of the things I was talking about.*

This time around? I got an email yesterday from ASQ HQ, six days after the post went down, with a partial explanation. The heart of it is a reminder that myASQ is the wrong forum for complaints about procedures that aren't working. The email suggested that I contact this office or that officer instead.

Public or private?

In principle it makes sense to specify that this channel is only for this kind of communication, but in this case I wonder. If I contact an office or an officer, the ensuing discussion will be private. But when the topic is a failed procedure—and a procedure that affects many people—I think the communication really has to be public.

The point is that people are already discussing the issue. When I asked, "Was anybody else as confused as I was?" it was a rhetorical question. I'd already talked to people in other Sections who had the exact same experiences and the exact same questions. So in a sense I was asking my questions on their behalf as well. In my reply to the email from HQ, I tried to make the point that a lot of people in the Society are already whispering in the corners about this very topic. If ASQ came out and addressed it publicly, they would silence a lot of whispering. And they would give members a renewed confidence in ASQ as an organization. It would be a win all the way around.

Of course—I tried to reassure the officer who wrote to me—discussing the issue publicly never means naming names! No one expects the Society to say "It's all the fault of John Jones and Fred Smith." But it is always possible to say "A Member misunderstood the procedure and therefore did X; then an Administrator followed up using the wrong template and did Y. We are currently investigating how to error-proof the process so that it is impossible to make those mistakes again." And, as I say, it would boost confidence to do so.

I don't know if I was persuasive or not. I guess we'll see.

Where do we report bad news?

Ultimately that's the important question: Where do we report bad news?

• We have to report it somewhere, because that's what we Quality professionals do.
• ASQ has urged us for years to channel all our communications into myASQ, so my first guess is that that's where we should report administrative failures like this one. 
• On the other hand, if myASQ is the wrong place to report bad news, then that's fine too—so long as we know what channel to use instead. And for practical reasons, it should be a more-or-less public channel, one that other people can consult. Is there another channel inside ASQ with these features? Or how about LinkedIn? Facebook? The New York Times? 

I'm ready to use any of them, so just let me know. 

__________

* Yes, I downloaded copies of the Terms of Service document—with date stamps—before and after, just to be sure.

           

Doesn't anybody know how to play this game?

Quality professionals have to know at least something about a lot of different fields. 

• We don't have to be logicians, but we have to know enough logic to work through a 5-Why analysis or to build a cause-and-effect diagram. 
• We don't have to be statisticians, but we have to know enough statistics to manage processes with a control sheet, or to sort failures with a Pareto chart. 
• We don't have to be experimental scientists, but it helps if we know something about the design of experiments.

We also have to understand organizational behavior and management. Most of the improvements we identify can't be implemented without the help of other people, generally people who don't report to us. So we have to know when and how to introduce changes so they can slide into place with the least disruption. It helps to develop a feel for the organization we work in, so that our colleagues in other departments are disposed to support us rather than to block us. 

That's why I was so startled at the very casual way I learned that ASQ had recently updated the procedure to elect leadership committees (or "Boards") for local geographical sections.

• I first heard an announcement (with no special urgency) that there would be a training class on the updated procedures on July 15. I was going to be out of town that day, so I made a point to listen to the recording as soon as I got back.
• Sure enough, I listened to the recording on July 23. It outlined a complex new procedure but specified no deadlines.
• That evening there was a general leadership meeting, and someone asked what the deadlines were for the new procedure. No one in the meeting knew, so the organizer took an action to find out and let us know.
• Only the next day, on July 24, did the meeting organizer send us a screenshot that the first deadline for the new procedure had been on July 8—more than two weeks earlier!

The discovery that we were already late caused a little consternation among our regional leadership. Our contact in leadership found and supplied two documents to clear up the confusion. Unfortunately they disagreed. One said to hold elections for two weeks, starting in October, with a slate of candidates compiled in August. The other said to hold elections for four weeks, starting in September, with a slate of candidates compiled in July.

Some time later I tracked down a copy of the procedure document itself. Turns out the new version had been released back in February, and the first deadline for the election process is actually July 1. But the first time anybody was told about it was in a training class two weeks later that didn't specify deadlines. As I say, the notification was very ... relaxed.

OK, you might say, so it was a simple bureaucratic mix-up. No big surprise. These things happen all the time. Why should it matter to you so much?

And my real answer is an emotional one, not a reasoned one. But we're ASQ! We're supposed to know better than that. We're supposed to be better than that!

But are we? On reflection it's not so obvious, for at least two reasons.

First, ASQ is an organization like any other. It has to hire staff, like any other. ASQ's members are Quality profes­sionals, but there is probably no require­ment that the Society's profes­sional staff all spend a certain number of years in the Quality trenches before coming onboard. Such a require­ment would be, at the very least, highly unusual; it would also force ASQ to pay well above market rates for office staff. 

Second, even seasoned Quality professionals sometimes make bone-headed mistakes. I know I have.* What we hope for, when working with Quality professionals, is not that they be perfect, but that they be ready to examine their own mistakes openly and dispas­sionately in order to learn from them. And naturally we hope also that they bring with them the memory of past lessons learned, perhaps in the form of a checklist or procedure, so that they don't have to learn from any particular failure more than once. But to ask that they never fail is simply to ask too much.

Maybe ASQ can carry out a root-cause analysis on the mistakes in this roll-out, and then assign long-term corrective and preventive actions. In other words, maybe ASQ can use the current muddle to improve the process for handling updated procedures. That kind of outcome could turn today's jumble and disorder to advantage. Let us hope.

__________

* Tip of the hat to the memory of the great Graham Chapman, in "The Mouse Problem."     

           

Can a Checklist Replace Your Audit Program?

A couple days ago, Quality Magazine published my article, "Can a Checklist Replace Your Audit Program?" It's their article now so I won't post the text of it here, but you can find it by following the link. I hope you find it useful! 

 

Quality in hard times

In last week's post, I remarked on the risk that mismanagement can teach employees not to care about the quality of their work: specifically, if employees are penalized for doing good work, they will quickly learn not to repeat the experience. A reader commented that this outcome doesn't even require active mismanagement; the simple failure to reward good work can do damage by itself. She wrote that the absence of any "consistent opportunity for reward can create a 'well, why bother then?' atmosphere." And of course she's right. I suspect many of us have experienced contemporary examples of exactly this dynamic at one point or other. In further discussion, she gave me a concrete historical example from the history of the British Royal Navy, whose officer corps had become so crowded during the nineteenth century that no further promotions were possible. Nor were there a lot of wars for the Navy to fight, once Napoleon was defeated in 1815. So with no chance for fighting and no hope of promotion, many officers stopped caring about their work and the corps began to rot.  

'What comes next? They become careless of themselves; most of them take to drink, more or less, for that is a habit almost impossible to wholly abstain from, while leading that kind of monotonous life; from that they become despised by their seniors in the service'.... The diary of a seaman in the Leander in the Pacific in 1863-5 notes seven officers court-martialed for drinking, not counting the chief engineer who died of it.*

Not officers, but you get the idea.

But this observation raises a further question. What does reward mean? Are rewards always money? If yes, we have a problem, because no company can count on consistent profitability, year in and year out. One year, somehow, times are going to be tough. One year, somehow, there won't be any extra cash on hand to distribute rewards for even the best service. And it would be painful to think that hard times might destroy the incentive for quality work.**

I have worked for companies that hit hard times, and my experience is that we continued to do good work. But everyone's case is a little different. There were a couple of people who were just there for the paycheck, or who had financial obligations that couldn't wait: they left promptly for greener pastures. And there were a couple who were laid off as the company retrenched. But many of us stuck it out, and continued to do the best we knew how.

If I ask myself Why? I see a number of factors. One is that we had already built up an ésprit de corps from working together in better times, so that none of us wanted to let down our colleagues. We weren't supporting The Company (which is more or less an abtraction), but we were supporting Larry in the next office, and Frank down the hall—people with whom we might bicker about little stuff, but whose work we still respected.

A second factor is that management communicated with us openly, and—so far as we could tell—treated us fairly. They rolled out information in All-Hands Meetings, so that everyone got the same message at the same time. And they let us know, "Here's what we can do for you today; here's what we want to do for you but can't right now; here's how we plan to get there." Knowing that we were all in the same boat made a difference. Knowing that we weren't being played for suckers made us willing to do our part.  

We discussed similar points in this blog last year. Mostly people like working together, so they are willing to cut the company some slack when things go wrong. But people can think for themselves, so they want you to tell them the truth. (And they can usually tell if you are lying!) People need you to give them honesty, respect, and simple justice every single day—which is far more often more than they ask for a raise or a bonus. So if you (as manage­ment) are willing to fulfill your obligations to give your people honesty, respect, and justice—even when you can't back those up with more money—they are generally willing to return the favor by continuing to fulfill their obligations to you. 

Thus can you ask for quality work even in hard times. But only thus!

__________

* From N.A.M. Rodger, The Price of Victory (New York: W.W. Norton & Co., 2024), p. 75. I thank Terri Williamson for raising the point out of her study of the Royal Navy, and for providing the citation. 

** The point would become even more acute in case the economy as a whole were ever to enter a contraction spiral, because then employees couldn't just jump to a profitable competitor. For explorations of this possibility, see e.g. the research of J.M. Greer, for example here.     

           

Where do you start? — take two

Last week, I wrote about employee engagement—and espcially about the research that suggests a strong correlation between employee engagement and business performance. But while I was writing, my mind started playing with the idea to see where it would lead.

In the first place, while the article that I reviewed was careful to assert no more than a correlation between engagement and performance, I reflected: If there were a causal connection, which direction would it point? Is it more likely that good business performance would cause employees to feel like they understand their jobs and have the tools they need? Or is it more likely that employees who do understand their jobs and do have the tools they need will perform well in the marketplace? Pretty clearly the second one is more likely.

In the second place, the article makes it clear that the measures for employee engagement are actionable. If your employees feel neglected and unengaged, there are concrete things you can do to improve the situation: make sure they know what is expected of them, make sure they have the tools that they need, praise them when they do well, and so on.

Then suddenly I thought of an article that I posted here four years ago. At the time, I asked: if you have just taken over responsibility for the Quality system in an organization that needs reform, where do you start? At a practical level I stand by what I said then, that internal system audits are the only way to get objective information about what's missing. But if you have several parts of the organization that need attention, where should you focus first?

Based on the Gallup results, I think the answer may be: Management.

After all, IF it is at least plausible that improved engagement drives higher quality work and improved results, and IF management can take concrete steps to improve engagement, THEN doesn't it seem likely that trying to improve performance without those concrete steps by management will be an uphill battle all the way? We have already seen that treating your people with justice and respect can be an important form of preventive maintenance to make sure the organization functions the way it should. So it stands to reason that if you want to reform or reinvigorate an organization, those are the points you should check first.

Now I want to draw a distinction. If you have to address "Quality issues" throughout an organization, that can mean two different things.

• Case 1 is a case like the Riverville plant that I discussed last month. The people there wanted to do a good job, but they had a different definition of how to get there than the managers of Octopus Enterprises who had recently acquired the place.
• Case 2 is more severe. It's the kind of place where people don't care about doing a good job, and just want to get through the day.  

In Case 1 people still have the right attitude, so I think the core issue is one of communicating how the new system will help them achieve the same good results they are already trying to achieve with the old system. Ideally, the new system will make things easier, though as we saw in the case of Riverville, the transition can be tough.

But Case 2 is brutal. And there I have to ask, How did things ever get this bad in the first place? My assumption—and I admit it is just an assumption—is that most people start off wanting to do a good job. If that's no longer the case in a particular workplace, my first guess is that the employees have been taught not to care about doing good work by management practices that penalize them when they do. The other possibility is that management inadvertently hired one or two bad apples, and then failed to remove them before they succeeded in corrupting the others. Either way, it is unlikely that the rest of the organization will reform until things are improved inside management.

And really, ... how would you reform an organization's Quality focus without insisting on Quality in management? I think it would have to be like the old Internet meme: The beatings will continue until morale improves.

CAVEAT: I have never had sole responsibility for turning around the Quality performance of an entire organization. And I know that ideas which make sense at my desk are often wrong in the real world. 😀 So I might be wrong about this too. If you have experience that leans in a different direction, please leave a comment to tell me about it!  

           

How much does engagement matter?

We’ve talked about “engagement of people” as a fundamental Quality principle, and at a superficial level it sounds obvious. Disgruntled employees are distracted by being unhappy, so they are likely to turn out bad work; happy employees are not distracted, so they can focus on what they are doing and turn out good work. On the other hand, the world is full of simple formulas that sound right but don’t stand up to empirical investigation. What about this one? Is it true? Or is it just an agreeable fairy tale?

In fact, the correlation between employee engagement and business outcomes has been the subject of fairly extensive research over the years. Last month I ran across a recent study by the Gallup organization, called “The Relationship Between Engagement at Work and Organizational Outcomes: Q^12® Meta-Analysis: 11th Edition” This report dates from 2024, and it is part of an ongoing project by Gallup. There are references in the notes to how the results have shifted since the previous edition. (Answer: not much.)

Briefly, what does the report say?

• The authors define a way to measure employee engagement, and then they identify eleven different ways to measure business performance. 
• They review a whole lot of companies, covering over three million employees. 
• And they find a consistent correlation: the companies whose employees feel the most engaged also have the best performance.

Why do you care?

• I’m glad to see the correlation with performance. Yes, I expected it. But it’s nice to see that it’s true.
• I like the way they measure employee engagement, because the metrics are actionable. I’ll explain what I mean in a minute.

Tell me about the method

Sure. The research pulled together 736 studies across 347 organizations in 53 industries, located in 90 countries; in total they reviewed 183,806 working groups, containing 3,354,784 employees. [See p.4 of the report.] This paper itself is a “meta-analysis,” a statistical method to combine studies of different sizes in order to filter out any idiosyncrasies and make the results comparable. The study establishes correlations but does not directly address causality. [See p.17.] In some cases, the authors deliberately omitted data from 2020, when effects related to COVID-19 clearly overwhelmed the effects that they were studying. [See p.18.] Overall, the findings “show high generalizability across organizations…. [M]ost of the variability in correlations across organizations was the result of sampling error, measurement error or range restriction in individual studies.” [See p.28.]

Fine, what were the numbers?

The study rated organizations based on their employee engagement scores. Then they calculated their business performance on each of eleven measures. Then they compared the business performance of those companies with high engagement against the business performance of those companies with low engagement. And they calculated the percent differences in the two sets of rankings. Here is the key table of results: [See p.5.]

Median percent differences between top-quartile and bottom-quartile units were:

• 10% in customer loyalty/engagement
• 23% in profitability
• 18% in productivity (sales)
• 14% in productivity (production records and evaluations)
• 21% in turnover for high-turnover organizations (those with more than 40% annualized turnover)
• 51% in turnover for low-turnover organizations (those with 40% or lower annualized turnover)
• 63% in safety incidents (accidents)
• 78% in absenteeism
• 28% in shrinkage (theft)
• 58% in patient safety incidents (mortality and falls)
• 32% in quality (defects)
• 70% in wellbeing (thriving employees)
• 22% in organizational citizenship (participation)

The paper spends many pages explaining the method and the calculations, but this table is the payoff. Companies with high engagement have 78% less absenteeism than companies with low engagement? Wow. Where do I sign up?

Wait—how do they measure employee engagement?

I thought you’d never ask.

The metric is something developed by the Gallup organization over years. It’s called Q¹², and it is based on a list of thirteen questions which the employee rates from 1 to 5. Twelve of these questions are very specific and concrete; one of them is a general evaluation. Here is the list: [See p.12.]

• Q00.  (Overall Satisfaction) On a 5-point scale, where 5 means extremely satisfied and 1 means extremely dissatisfied, how satisfied are you with (your company) as a place to work?
• Q01. I know what is expected of me at work.
• Q02. I have the materials and equipment I need to do my work right.
• Q03. At work, I have the opportunity to do what I do best every day.
• Q04. In the last seven days, I have received recognition or praise for doing good work.
• Q05. My supervisor, or someone at work, seems to care about me as a person.
• Q06. There is someone at work who encourages my development.
• Q07. At work, my opinions seem to count.
• Q08. The mission or purpose of my company makes me feel my job is important.
• Q09. My associates or fellow employees are committed to doing quality work.
• Q10. I have a best friend at work.
• Q11. In the last six months, someone at work has talked to me about my progress.
• Q12. This last year, I have had opportunities at work to learn and grow.

I said above that I like this way to measure engagement, because nearly all of these points are actionable. The Gallup organization deliberately chose to ask about issues that individual managers can correct! So if someone carries out a Q¹² survey at your place of work and your department scores low in this or that area, mostly you (if you are a manager) can do something about it. Make sure your people know what is expected of them; make sure they have the tools that they need; make sure they are doing work they are suited for; praise them when they do well; and all the rest. These are part of your job as a manager anyway. The use of this metric means that if you carry out all these tasks conscientiously, you can overcome any bad score related to engagement. You can improve. And we have already seen that quality in management is a critical benefit to the whole organization. 

And in other news ….

Meanwhile, for those who thought the whole point was obvious from the beginning, take heart. In other news, researchers have carried out an expensive study to prove that the best bait for mice is cheese.*      

__________

* “During a six-month period in 1973, The New York Times reported the following scientific findings: A major research institute spent more than $50,000 [$377,390 in 2026 dollars!] to discover that the best bait for mice is cheese….” From Jerry Mander, Four Arguments for the Elimination of Television (New York: Quill, William Morrow and Company, 1978), p. 53.

           

Auditing a system that uses AI

We’ve talked before about some of the Quality issues related to the use of artificial intelligence (AI) technology, but mostly in an incidental kind of way.* But the adoption of AI by many companies seems not at all incidental. So it is fair to ask how the introduction of AI is going to affect core Quality functions. How will AI change—say—auditing?

It's a good question, and there have been a couple of attempts to answer it in the past year. Last October, Elisabeth Thaller and Jorge Bravo Carreño wrote an article "When the QMS Thinks for Itself" for Quality Progress. Then just last month the ISO 9001 Auditing Practices Group published a formal guidance paper on "Auditing a Quality Management System that uses Artificial Intelligence (AI) systems." It is perhaps not surprising that the article and the guidance paper repeat the same basic points,** and they are pretty much the same points you would make if someone asked you about the same topic. Nothing here is unexpected.  

What makes this consistency possible is that ISO 9001 is structured to apply to any kind of work. So the basic questions are always the same. What are your processes? What are your risks? What are your training requirements? And are you getting the results that you want?

The article in Quality Progress identifies five main thoughts to keep in mind when planning an audit in an organization that uses AI, and the sugges­tions in the APG guidelines can almost be mapped directly onto the same list. The five main thoughts are these:

1. Identify where the organization is actually using AI. ("AI may be hiding in plain sight.")
2. Use the process approach to identify inputs and outputs in the normal way. ("The process approach still applies—even when the process thinks for itself.")
3. Identify risks. ("Risk-based thinking isn’t optional.")
4. Identify training needs, responsibilities, and authorities. ("Competence and oversight are essential.")
5. Identify results and look for objective evidence. ("Focus on evidence, not the shine.")

And while I'm certainly not going to quote all ten pages of the guidance document in a blog post—you can download the whole thing for free by using the link above—let me quote just a couple of items that fit under each of these five headers, to show you the overlap.

Identify where the organization is actually using AI.

• When preparing the audit, the auditor should:
• Identify whether the organization uses any AI system type for any of its processes within the scope of the QMS.
• Determine the specific function(s) of these processes within the QMS, and
• Recognize assigned responsibilities in relation to these processes. [§2, p.4]

Use the process approach.

• Where AI system(s) are used within operational processes, has the organization ensured that their use supports the achievement of intended results?
• Are such processes monitored, and are changes related to the use of AI system(s) controlled?
• How does the organization ensure the consistency, correctness, and reliability of AI outputs? [§3, cl. 8, p.7]

Identify risks.

• Has the organization considered the implications of the use of AI system(s) when determining its internal and external issues?
• Has the organization identified statutory and regulatory requirements in relation to the use of AI systems, such as those related to data privacy and information security? [§3, cl. 4, p.4-5]
• Has the organization determined and addressed risks and opportunities related to the use of AI system(s)? [§3, cl. 6, p.6]

Identify training needs, responsibilities, and authorities.

• Are adequate resources available to maintain and update AI system(s), including access to technical expertise, data quality management, and cybersecurity support?
• Is the organization able to demonstrate that personnel who use, manage, or oversee AI system(s) meet the determined competence requirements? [§3, cl. 7, p.6]
• Does top management take accountability for ensuring that the AI system(s) within the QMS support its effectiveness and achievement of intended results? [§3, cl. 5, p.5]

Identify results and look for objective evidence.

• Are customer satisfaction trends evaluated for potential impacts arising from AI-enabled processes or interactions?
• Is internal auditing addressing the effectiveness of AI-influenced processes?
• Is information related to the performance and effectiveness of AI systems an input into management review? [§3, cl. 9, p.8]
• Are nonconformities or complaints involving AI system outputs recorded, analyzed, and addressed? [§3, cl. 10, p.8]

There shouldn't be anything shocking in this list. One of the strengths of ISO 9001 is precisely its flexibility. And that flexibility comes from seeing all kinds of work through a lens that highlights certain common features. AI is just one more topic viewed through the very same lens. This is why I said above that you would have come up with the same points if you had taken the time to work through the standard in detail.

But most of us are too busy with our day jobs to do that, so it's convenient that the Auditing Practices Group has done it for us. The next time you have to audit an organization that uses AI in its management system, check out the guidelines document to see if any of the suggestions help you. 

__________

* In this post, I asked what the word “Quality” really means in the context of AI. In this one, I discussed how AI tools might be able to enhance training records. 

** Partly this outcome is unsurprising because the same person—Elisabeth Thaller—was the lead writer for the article and the chair of the committee that wrote the guidance document. Chris Paris published an article taking the APG committee to task because most of them have no special background in AI. But it appears from her LinkedIn profile that Thaller does have at least some background.        

           

Sampling bias

A friend forwarded this cartoon to me a couple of weeks ago. It seems to come from the website Sketchplanations, drawn by Jono Hey. You can find the original here.  

Of course statistics are central to the Quality discipline, so we need to understand how sampling works. And I think the cartoon makes a serious point in that regard, rather deftly.

It's also funny, and high time too!

            

Can you be TOO optimized?

There's an old story about a wise man that has some relevance for bloggers as well.

Once upon a time there was an old man renowned for his wisdom. Whenever a discussion got tangled and tempers rose, he always told a parable that resolved the question perfectly. Finally a little boy asked him, How could he unfailingly pull out a story each time that summarized the issue so well? "Ah," said the old man, "you don't understand my method. I don't ransack my brain looking for a parable to match each new conversation. Rather, I introduce a topic of conversation for which I already have the perfect parable."

Blogging is a little like that. We bloggers are sure we have the right answer, whatever the question. And by the time you read a post all the way to the end, we've usually made the point pretty convincingly. It almost looks like we're right about everything. (Or I hope it does. 😃) But then, ... who picked the topic for the post in the first place? We did! So of course we're going to pick topics where we think we can win the argument.

But I've been thinking recently about a plant I encountered a few times during my career, where things seemed slightly off-kilter but I could never tell why. I didn't work there, and they weren't part of my normal responsibilities. But I visited a couple of times, to support audits or for other reasons. The audits turned up findings, as audits always do; and the findings were always addressed in the usual way. Somehow, though, I never got the sense that we uncovered anything more than surface issues. It felt like there was something more basic that we were missing.

This plant was located in a town I'll call Riverville, so I can refer to the place as the Riverville plant.

Recently I've been formulating a hypothesis that might account for part of the situation there. So I want to describe the hypothesis, and ask whether you think this could pose a problem in real life? I won't go into a lot of details about how things were at Riverville, if only so that I don't identify it too plainly. But let me know if you think the circumstances I describe could cause some related mis-fires.

Then if it turns out that I'm all wrong, I can go back to questions where I (think I) already know the answer in advance. 😀  

The Riverville plant

Visiting and auditing

The Riverville plant had been in operation for many years. The people were deeply competent, and committed to their work. Also, because these things happen, the plant had undergone multiple changes of ownership throughout the decades it had been in business. Sometimes the new owners ran it as a standalone operation. But occasionally the new owners wanted it to integrate into a larger ecosystem of other businesses that they had also acquired. When I knew them, Riverville had been recently acquired by Octopus Enterprises (not their real name).

Despite the acquisition, though, the plant never felt like the other Octopus plants that I had seen. When visitors started to quote a lot of the normal Octopus manage­ment terminology, I thought I could see the Riverville old-timers smile to them­selves, as if to say, "Sure, we'll play along until you go home. But that's not how it really works." When I audited, I kept turning up forms that I'd never seen at any of the other Octopus plants. "Don't you use the standardized Octopus forms?" "Oh sure, sure, ... mostly we do. But that one's an exception because of the special way we handle [topic] here. It's authorized, though. Let me show you the procedure document!"

And there was always a procedure document to back it up. There were so many procedure documents, in fact. Sometimes I thought it would be an interesting project to collect all of them and paste them together, to try to get a picture of how the whole plant functioned. But of course I couldn't. As I said, I never worked there regularly, so in the normal course of things I would have had no business even raising the question. And in the absence of a total breakdown of their system—which never happened during the time I was familiar with the plant—I could never have made an argument to justify the time and effort. Besides, mostly the folks at Riverville were able to provide the right management deliverables back to Octopus headquarters when asked, and mostly they were able to get their regular work done in a routine way.

Fully optimized

As for those forms? Oh, the forms were awesome! Someone had put a lot of time into these. There were links to data sources on the internal network, so that when you needed to fill in fields 5-15 you just clicked a button on the form. Then you added a little more data by hand and clicked another button, and right away the form routed itself to the next person in line. None of this was managed by some overall data management tool. It was all hard-coded into the forms with URLs that pointed to archive repositories where project information was stored in suitably-structured Word and Excel files. 

Rebuilding the engine in flight.
This is not recommended.

In fact, the forms were the key to my recent hypothesis. I think the Riverville plant was too optimized to change! They had worked together for so long that they had optimized every single transaction in a large and diverse factory. But they optimized them all according to the outlines of the old Riverville management system. If a new owner let them run themselves, and just showed up at the end of the year to collect the profits, there was no harm done. But when Octopus Enter­prises tried to make them integrate with the Octopus manage­ment system, the only way they could do it was to run two systems in parallel. They couldn't stop using the old forms, because everyone relied on them. They couldn't stop using the old data­bases, because the forms pulled data from them. They couldn't stop working in the old way, because everyone in the company depended on every step of the old system. To replace the old Riverville system with the new Octopus system would have meant rebuilding the airplane engine in flight. 

I was never able to dig deeply enough to prove that the Riverville personnel were running two systems in parallel—I mean, as opposed to adapting the Octopus system by adding "just a couple" of the old Riverville forms—but it would make sense of my general observation that somehow things were a bit off. As we discussed last week, running the Octopus management system as a Potemkin system would have been a perfectly natural response, if the Riverville management felt that they were being forced to make changes that didn't help them any. 

The problem, of course, is that working in two parallel systems takes twice as much effort as working in one. Also, if you have data in two places, it is almost impossible to keep them both consistent. Whatever options Riverville faced, this one cannot be called The Easy Choice.

End of the Bronze Age

My thinking about Riverville has been supported in an interesting way by an article that appeared in LinkedIn around the end of March. The author is Marco Nutini, a risk-management expert from Brazil, and the title is "1177 BC Called. We're Not Listening." 

It's a fun article, and not long. Read it, if you have a few minutes. Nutini starts off summarizing his thesis with admirable brevity:

In the late Bronze Age, the Eastern Mediterranean was a marvel of interconnection. Egypt, the Hittites, Mycenaean Greece, Ugarit, Cyprus — linked by trade routes, diplomatic marriages, and supply chains that moved copper, tin, grain, and luxury goods across vast distances. It was, by the standards of the era, a globalized world.

Then it collapsed. Not from a single cause, but from all of them at once.

His point is that the highly interconnected world of the late Bronze Age was optimized for current circumstances. Assuming that the world stayed at peace, that the rain fell reliably, and that no major population centers were wiped out by earthquake or volcano—assuming all that, the world ran just fine. But when those assumptions failed—when drought, famine, and earthquake came, when cities revolted and the Sea Peoples were landing on every shore—the finely-calibrated economy of the late Bronze Age could not withstand the shocks. It fell apart, collapsing into the Greek Dark Ages. Optimization for normal times is not the same thing as resiliency in trying times.

Admittedly, having your factory bought by Octopus Enterprises shouldn't be as devastating as having your seacoast raided by pirates, or your island explode. But either way, I think there is a risk that over-optimization can make you fragile and unable to adapt when circumstances change.

If I'm wrong, feel free to tell me so!