"How's our Quality?"

One of the risks in a mature Quality system is that there are a lot of details to master. So when you've mastered all the details, you think you know what's going on. But sometimes when all those trees crowd together you can lose sight of the forest.

I remember once, years ago, I worked for a company that had a very mature system. It seemed like they had metrics defined for everything but the coffee pot. When I first started there I had to learn the whole system pretty quickly, and my boss focused on the fine-grain details. Then suddenly we reorganized: instead of reporting to a functional expert located far away, now I reported to the General Manager in my own location.

For my first meeting with him, I brought along all the charts and metrics that my former boss had always wanted to review, but the GM waved them away. Instead, he asked me, "So how is our Quality?"

I fumbled for a minute and started to explain, "We measure that in a lot of ways. Now this chart, for instance, shows that ...."

The GM cut me off. "How's our Quality?"

I couldn't tell what he wanted to know.

"Look, I know that the guy you used to work for asked for all these dozens of metrics. He used to present his charts to me and I couldn't make head nor tail of them. I just want to know how we are doing."

Whatever answer I gave didn't satisfy him. So when I got back to my desk I pulled out all the metrics and tried to look at them from a high level. I realized that they fell naturally into three buckets:

• metrics related to our product performance and customer satisfaction;
• metrics related to how well we complied with the formal procedures mandated by Headquarters;
• and metrics that tracked internal administrative activities—these weren't as important as the other two, but I had inherited them as part of the system so I still calculated them.

The other thing I realized was that the performance inside each bucket was pretty consistent. When I saw that, I knew how I was going to report the results to our GM.

Next week was my next meeting with him. And sure enough, he started by asking, "How's our Quality?"

This time I had an answer. "Our product quality is great, and our customers are very happy. Those metrics are all green. Our process adherence is lousy. Those metrics are all deep red. And our internal functions are kind of wishy-washy, not great but not terrible. You can call those yellow, if you like."

The GM grinned. "Thank you. That's what I was looking for. Also it's pretty much what I expected. Now go fix the ones that aren't already green."  

"Aye aye, sir." 

We communicated a lot more easily after that. And I learned, never lose sight of the big picture. Ultimately all those details are there only to support and flesh out that big picture. But if you can't see the big picture, you don't know where to start work.

      

How do you manage "special responsibilities"?

One of the first things you learn when you write formal procedures is, Don't use personal names! 

Let's say you are writing a calibration procedure. As long as anyone can remember, Dave has been checking the calibration of all your tools on a rolling cycle, so he checks each tool twice a year and records the results in an Excel sheet. Finally you get around to documenting what he does in a procedure. What you must not write is, "Dave checks the tools twice a year on a rolling cycle." Why not? Because if you use Dave's name in the procedure, sure as anything he'll move to Tahiti next month and you'll have to update the document to name someone else.

So you replace Dave's name with a functional title, and write: "The Calibration Coordinator checks the tools twice a year on a rolling cycle." Note that this isn't Dave's job title. In many companies, Dave's job title is something like "Engineer 2" that can be correlated with his pay grade. And you certainly don't want just any old "Engineer 2" to do this job. The person who does it—Dave, or his successor—has to be qualified by knowing about calibration. That's why you create a special functional title just for the sake of this specific document, and you call Dave the "Calibration Coordinator."

As you write procedure documents for your operations, you'll do this several times—half a dozen, maybe more or less, depending on how complex or specialized your operations are. When you are done, you'll have a collection of functional titles that exist only in your procedure documentation, to account for specific roles your quality management system needs. These functional titles identify your QMS's Special Responsibilities. Of course in practice they just point to Dave and Pat and Sue and Sam, who handle the respective tasks. And when the auditor reads your Calibration Procedure and asks, "Calibration Coordinator? Who's that?" right away you say "Dave."

Then the auditor asks, "How do you know?"

Make a list 

One way to manage these assignments is to make a list. Seriously, it can be that easy. Just read through the formal procedures in your QMS, and write down every Special Responsibility they define. Then publish a List of Special Responsibilities (LoSR). For each entry in the List, identify the following four pieces of information:

• 
  Title of the role? (e.g., Calibration Coordinator)
• What document defines the responsibilities? (e.g., Calibration Procedure)
• Who is assigned to that role? (e.g., Dave Smith)
• Who is the Proxy, in case the regular assignee is out? (e.g., Pat Jones)

And that's it. The next steps are just as straightforward: 

• Make sure the List is formally approved by a suitable authority. (Since the list covers roles throughout your QMS, the "suitable authority" is usually your top management.)
• Formally identify the List as part of your QMS documentation.
• Maintain the List through formal change control, so that it is always up to date and so that the assignments are communicated to all stakeholders.

Enhance your documents

There's one loose end, but it is easy to tie up. Remember I said that not just anybody can be Calibration Coordinator? Spell out the qualifications for the role. 

An easy place to document them is in the same procedure document that defines the role in the first place. For example, your Calibration Procedure explains all the steps required to calibrate your equipment; so you can easily add one extra paragraph that spells out the necessary qualifications for the Calibration Coordinator. 

Then Dave's training records include an entry that shows how he met the qualification requirements. Maybe he took a class and got a certificate; or maybe he learned on the job from his predecessor, who sent an email when Dave was ready. Either way, the assignment and Dave's qualifications are fully traceable.

• The List of Special Responsibilites assigns Dave to his role, and references the document that explains it in detail.
• The document also lists the qualification requirements.
• And Dave's training records show that he meets those requirements.

If you have used other approaches you like better, leave me a note in the comments.

    

How small is "small"?

Last week, I wrote about an organization I once knew that accomplished amazing results with almost no formal systems. Among other things, I said, "When the number of people is small, and when the average tenure in jobs is high, you can accomplish a lot of good work following the Everybody-Just-Knows methodology." It sounds promising.

But how small is "small"? At what size do you lose the magic of "smallness" and have to start putting systems into place?

It's sooner than you think.

A friend of mine works for a retail firm with around sixty-five employees—maybe more now and then for short-term or seasonal work. It's a "small company" in anybody's book. But a little while ago they ran into a situation that proved the need for some formal systems.

What happened is this. The company generates product information sheets for their products. These sheets are used by several departments internally, and they are also made available to customers. Some time ago, the Social Media team decided to update these information sheets to make them "easier to read": in practice this meant adding some colorful graphics and rearranging the text so there is more white space on the page. For sheets with too much text to rearrange, they just deleted anything they found boring.*

But the Sales and Support teams, who interact directly with customers to resolve problems, urgently needed all that deleted text. So they in turn just kept copies of the earlier versions, and continued to duplicate those outdated versions when they needed more. At no point did anyone escalate the issue to the responsible manager to get a ruling on how to proceed, because no manager is specifically in charge of the information sheets. They could have gone to the General Manager, but no one wanted to escalate the issue that high.

When my friend told me this story I tried to discuss it pleasantly. But inside I felt like I was looking at one of those pictures you see in magazines with the headline "How Many Mistakes Can You Find?" (1) Nobody was formally responsible for the information sheets. (2) The people updating the sheets didn't know the full list of interested parties who used them, nor (3) what those parties required. (4) The updated sheets were not sent out for review before they were approved. And (5) it was possible for people to keep and use old sheets instead of having to use the current ones. There might be more issues if I think about it for a while, but that list is a good start.

I don't blame the company. No company starts out with all these systems on Day One, and it is usually through episodes like this that they learn they need more than they've got. Also I wanted to tread a little lightly because the company wasn't a client. They'd never asked for my advice—my friend was just telling me about having a bad day at work. So I made a few suggestions for her to take back to the office, to see if she could nudge things in the right direction, and left it at that.

But how is this any different from the team I described last week? If that team could do consistently good work with no formal systems, why shouldn't we expect the same results here?

The easy answer is that there are no tricks in the world that will guarantee you do good work. Even a formal Quality Management System won't guarantee that. At most, the Quality techniques will help you avoid a lot of familiar known mistakes that otherwise recur over and over. So while I say that a small team can do good work without formal systems, that doesn't mean they will.   

Beyond that, there are two or three differences between the Repair Center I described last week and my friend's retail store. That is, there are two differences which definitely bear on this question, and a third which might.

1. The retail store is small for a company, but it's nearly ten times the size of the Repair Center.
2. The Repair Center all did the same kind of work, and they all sat together in one big room. The retail store has multiple departments that are spread out across the facility.
3. Everyone in the Repair Center was an "old-timer." Most of them had worked in the same job for over a decade. The retail store has more of a mix of age and experience levels.

The retail store's problem with information sheets arose largely because some people (the Social Media team) didn't know the needs of others (the sales and support staff). The first two points above mean that could never have happened in the Repair Center. Everyone there knew what everyone else was working on—and what they needed, and almost what they were thinking—because they all did the same task and they had worked together for so long. The last point, about seniority, might have helped too, because one of the benefits of long experience is that you've already had plenty of years in which to make foolish mistakes: after a while you should have already committed most of the relevant mistakes, and learned from them.

So when I say that a small team can do good work without formal systems, there are a lot of other conditions that have to be in place instead. The team needs to be really small, small enough that all-to-all communication is easy and immediate. They have to be knowledgeable and experienced. There are personal characteristics that they have to have, as well: conscientiousness, focus, and a host of others. It's possible, but it's not easy.

If you want to grow larger, consider starting to put formal systems in place.

If someone retires, look closely at what he used to do. Often someone will start doing basic document or system maintenance on his own time, just because he sees that it needs to be done. When he leaves, it may be time to formalize it.

And remember the other extreme, just to keep everything in perspective: that you can have sophisticated formal systems in place and still make alarming mistakes. 

__________

* That's probably an exaggeration; but in light of what came next it might as well have been true.   

     

Should Boeing get certified?

Last week we talked about Boeing's recent quality problems. I made the point then that no Quality Management System can ever prevent all possible failures, but that a good QMS can reduce the incidence of failures dramatically. In addition, a QMS can tell you how to react when a problem occurs, to mitigate or prevent bad consequences. And clearly Boeing's QMS has helped keep things from being worse than they have actually been.

But building aircraft is hard. ^{[No citation. I assume this is obvious.]} So it is fair to ask if Boeing's QMS is already as good as it can possibly be, or is there room for improvement? A second question, continuing the thought, is whether Boeing's QMS could benefit from formal certification to AS9100? It is well-known that Boeing supports the use of AS9100 and requires certification of all its suppliers, but does not hold a certificate of its own (although some of its subsidiaries do). Readers may remember that I discussed this irony a little over a year ago: in brief, Boeing made an internal business decision not to seek certification, and they have taken some criticism for it. And many companies seek formal certification of their QMS'es as a tool to drive continual improvement. Could Boeing benefit?

Clearly I'm not close enough to know for sure, and in any event Boeing has not asked for my advice. All I can offer are a few thoughts.

Any process can be gamed

We know from the outset that Boeing's existing QMS is already very good, or things would be a lot worse than they are. And Boeing says overtly that their QMS is "based on AS9100." So, many of the pieces should already be in place. But any system can be corrupted—any process can be gamed—if the underlying business culture doesn't support it. And there have been a lot of articles in the news lately about Boeing's business culture. (For just a small sampling of articles published in the last week, consider this, this, and this from LinkedIn, or this from Australian Broadcasting News.)  

Here's an example to explain what I mean when I say that any process can be gamed. Consider checklists. A checklist is one of the simplest process controls there is, and it is amazingly powerful. Introducing checklists into the practice of emergency medicine has saved countless lives. What's more, it almost seems like a checklist is too simple to game. But let me tell you a story.

One day I was auditing a warehouse. They had a forklift, and they explained they had a requirement to do a safety check every morning before using it. 

I asked, "How do you make sure that check is done every morning?" 

They explained, "The responsible operator has to fill out a checklist, and then he signs it and dates it."

"Great. Can I see the checklists for this week?" (It happened to be Wednesday.)

"Sure. Here." They handed me a clipboard.

So I looked through the papers on the clipboard. Here was the checklist for Monday, filled out and signed; then for Tuesday; then for Wednesday (the day of the audit); ... and then—wait, what's this? Next came the checklists for Thursday and Friday, dated in the future, fully checked off, and signed.

Obviously they had filled out a week's worth of checklists all at once. Had they ever done a safety check on the forklift? Since I couldn't trust the checklists, I had no way to know.

That's what I mean by "gamed" or "corrupted." You can have the best processes in the world—on paper—but people will always find a way around them unless there are strong incentives not to. The best case is to have the kind of company that always prides themselves on doing things right, so that no employee—from the C-suite to the production floor—would stoop to getting around the rules.

Could formal certification prevent this kind of corruption? Maybe not, but it could make it harder. And that's because ….   

Don't lie to yourself

In the story I just told, I found the problem with the checklists by doing an audit. Since Boeing's QMS is based on AS9100, they must certainly do internal audits. In principle, that should be enough to catch any corruption of the system. 

But if you walk past something every day, after a while you don't see it any more. And if there are strong cultural incentives not to uncover certain kinds of sloppiness, you may come to find the sloppiness so normal that you stop questioning it. Even if you start with the best will in the world, it is possible to drift through the routine, day after day and year after year, until you end up lying to yourself.

This is bad. Don't let this happen.

One way to reduce the risk of lying to yourself is to bring in a stranger, who can look at your operation with fresh eyes. That's the service performed by an external auditor from your Certification Body. So while we've all met auditors who are a little goofy, and we have all gone through audits that felt pointless, nonetheless an external auditor sees with fresh eyes to give you an outside perspective. An external auditor is better placed than anyone internal to tell you whether your system retains its fundamental integrity. 

An external auditor can help keep you from lying to yourself.

Again, I don't know the true source of Boeing's recent troubles. I don't know if certification and external auditing would fix the problems. But they might help. They couldn't hurt. 

          

Life as a conductor

Last week I talked about the role Quality personnel have in an organization, even though they are not directly engaged in production or sales. And it put me in mind of a comparison that Rose Duncan made some weeks ago. (I've quoted Rose before, briefly in this post here.) Rose's image was simple and memorable, and it would have delighted me even more back when I was five years old and loved trains. 😀

She wrote:

Your quality management system / QSE is like a train.

You need an engine and carriages.

You can attach as many as you want: beautiful ones, compliant ones, expensive ones, simple ones, complex ones, unique ones, a few, or a lot...

But let's be clear:

The engine is the leadership.

Their job is to pull the train.

You [in Quality] are the conductor.

Her post went on from there, and mostly it was about leadership. Her main point was that without leadership, the organization goes nowhere.

But I was struck by the idea of Quality as the conductor. What I commented at the time was:

The conductor doesn't personally load the coal (or diesel). The conductor doesn't hop down into the yard to hook up the cars manually. The conductor doesn't choose what speed to go around a tight curve. The conductor doesn't personally do ANY of those things! But you've got to have a conductor.

But then I went to look up, What exactly does a conductor really do? According to Wikipedia, the tasks of a conductor include:

• Ensuring that the train follows applicable safety rules and practices
• Making sure that the train stays on schedule starting from the stations
• Opening and closing power operated doors
• Selling and checking tickets, and other customer service duties
• Ensuring that any cars and cargo are picked up and dropped off properly
• Completing en-route paperwork
• Directing the train's movement while operating in reverse
• Coupling or uncoupling cars
• Assisting with the setting out or picking up of rolling stock

And seriously? That doesn't look like a bad match, mutatis mutandis. Let's look at it again, more slowly.

• Ensuring that the train follows applicable safety rules and practices Quality regularly has to check that the organization is following all applicable regulations, including legal and contractual ones. Some of these checks might be carried out by a Facilities Department or a Safety Committee. And yes, strictly speaking legal or safety regulations might be out of scope for the QMS, depending on how it has been defined. But Quality has a clear interest in the topic of regulations in general.
• Making sure that the train stays on schedule starting from the stations Again, schedule maintenance might be most immediately managed by Project Management or Production Management. But if the organization regularly fails to meet its committed delivery dates, there's going to be a long talk during internal audits about why.
• Opening and closing power operated doors Remember that this is an analogy. If the train represents your system, then the doors are points in your process that accept inputs or release outputs. Presumably you have requirements that your inputs and outputs have to meet. Quality safeguards that those requirements are met, and therefore functions as a "doorkeeper."
• Selling and checking tickets, and other customer service duties Tickets are how you make sure that "passengers" (which Rose later identifies with your employees) are in the right places (roles / functions) at the right times. Quality checks that training requirements have been met, and that resources are adequate.
• Ensuring that any cars and cargo are picked up and dropped off properly This is again related to inputs and outputs, as above.
• Completing en-route paperwork Everyone knows we are all about paperwork. 😀 (Not solely, of course. But it is inevitably part of the picture.)
• Directing the train's movement while operating in reverse Under certain exceptional circumstances, when something tricky is being done, it is not unheard-of for Quality to be given—temporarily!—some kind of operational role for the duration of the special circumstance.
• Coupling or uncoupling cars ... and ...
• Assisting with the setting out or picking up of rolling stock I ran out of inspiration for these last two. No analogy is perfect.

So there you are. It's a fine analogy to explain to five-year-olds everywhere what a Quality professional does all day. We're conductors. And every train needs a conductor.

           

Is Quality really free?

A few weeks ago I was having lunch with a friend, and we were talking about work. She works for a small company struggling with growth, and when she started there her charter included implementing a Quality Management System. But there were always daily fires to put out first, and at this point she said she has shelved that project for the time. There are a lot more immediate things that have to be done first. Finally she sighed and said, "We just can't afford Quality right now."

This took me aback, just a little. Ever since Philip Crosby, we have all told each other that "Quality is Free." But how can that be? In fact, this slogan turns on the dual meaning of the word quality. On the one hand, quality just means doing things well. On the other hand, Quality [often with a capital Q] refers to a whole body of formal practices that organizations implement—a professional body of knowledge, with its own professional organizations and certifications. As I have said many times before (including as recently as last week), these two meanings are not the same.

Quality should be free, at the most basic level, because in a perfect world (where everyone always knows what to do) it should cost no more to do a job right than to do it wrong. In a more sophisticated sense, Quality is "free" because—even if it does cost a little bit more to do it right—you make that money back by avoiding the cost of rework or of replacing failures under warranty. This second line of thought is what justifies the cost spent on a formal QMS: yes, filling out the forms and following the procedures takes time (and time is money); and yes, you are paying inspectors and auditors who are not themselves engaged in production or sales. But without these forms and procedures and inspectors and auditors, your rework and warranty costs will eat you alive; so in the long run you are money ahead by having them.

"In the long run." That's the key. Or as one of my coworkers put it many years ago, "Therein lies the rub, bub." You have to have some time and money to spare first, before you can put the systems in place that will ultimately generate any savings.

Think of a paved road, as an analogy. Paved roads are a very basic—and very useful—piece of infrastructure. They enable the movement of human beings, the transportation of goods, and the delivery of services. Modern economies would be unthinkable without paved roads. For the economy as a whole, the net return on investment—the surplus of economic activity that they enable minus the cost of building and repairing them—is incalculably huge.

But paved roads don't grow on trees. For each paved road you see anywhere, someone had to decide where it was going to go; someone had to grade the land; someone had to lay out the road; and (only at the end of a long process) someone had to pave it. All of that required human effort, all of it took time, and all of it cost money. In an economy with no way to generate a surplus, no one can afford to pave a road.

And I think this is where my friend is, right now. There are just too many things that have to be done first, before a QMS will even be a meaningful concept. Maybe sometime in the next year she can move the inventory system out of Excel. Maybe she can regularize the purchasing activity. One thing at a time. And then in a while, when the company can afford it, she can go back to implementing a QMS.

Does that mean the company is doing a bad job in the meantime? Heavens, no! The company's day-to-day work is just fine, because the people who work there have mostly been there a long time and "just know" how to do it right. The overall level of competence across the team is very high. So yes, they are doing a good job today. The only problem is that their current methods aren't scalable, precisely because they depend on such a high level of competence. So if the company wants to grow, they are going to have to put systems in place. They are going to have to rely more on procedures and less on "just knowing." They are going to need a QMS.

Maybe next year.    

          

Integrating risk management with Quality

How does risk management relate to Quality? There's obviously some connection. If you don't think about risks when designing a product, there's a chance it might hurt someone when it goes out to the field. And if a product hurts someone, then it's a bad product—not a quality one. But when you set up a Quality Management System and a product development process, it's not always obvious how to integrate the consideration of risk. Is there some magic spot on the flowchart where you can drop a box called "Risk" and just have done?

Last week I wrote about risk evaluation, taking my lead from a LinkedIn post by Etienne Nichols that focused on the medical device industry (where he works). I was delighted to get a detailed reply from Edwin Bills which clarified some technical points and generously offered guidance to further reading. I wasn't familiar with Bills's work at the time, but I looked around and found that he has had a long and productive technical career in Quality and Risk Management, especially (but not only) with respect to medical devices. He has also published multiple articles. One recent article addresses exactly the question I've asked here.

This article dates from May 2023, not quite two months ago, and is titled "The Intersection Of ISO 13485 And ISO 14971 Under The Proposed FDA QMSR." (You can find it here.) It focuses on the medical device industry, which is not exactly my wheelhouse, so I won't try to comment on the technical details. (Besides, if you are curious about those details you should read the article itself.) But Bills makes a number of broader points that are very valuable.

The first of these points sounds obvious at first, but has deep ramifications: there is a difference between product risks and business risks. In exactly the same way, there is a difference between a standard designed for regulatory compliance and one designed for process management. 

• With respect to the first distinction, product risks are in general a lot more detailed than business risks, and require a much more granular response. Risk analysis at the business level might highlight the need for a new policy or a change in strategy; risk analysis at the product level could identify dozens or hundreds of potential risks, of which let's say twenty or fifty are actionable. Then each of the actionable risks has to be analyzed in detail to determine exactly how to prevent it (or at least mitigate its effects).
• With respect to the second distinction, Bills observes simply that the first edition of ISO 13485 (Quality management systems — Requirements for regulatory purposes) was explicitly based on ISO 9001:1994, because the 20-element structure of the latter was very useful in supporting regulatory compliance. By the time ISO 9001:2015 came out, ISO 13485 no longer copied its structure—because ISO 9001's process approach (together with the frequent use of the phrase "the organization shall determine") made the document altogether too vague to be much help in an FDA audit.

A second general point that Bills makes is that—to answer my question in the first paragraph—there is no magic spot on the flowchart where you can drop a box called "Risk" and just have done. Bills takes considerable pains in his article to trace out the interconnections between ISO 13485 and ISO 14971 (Application of risk management to medical devices). As always, you can find the details in his article (and he provides a map at this link). Here let me say simply that the connections are many and subtle; and while he is clear that you do not have to comply with ISO 14971 in order to comply with ISO 13485, it sure helps. Bills also walks his readers through the entire product life-cycle—from design through development and into production—to show exactly what role risk management has to play at each step of the way.

There is a third point that Bills brings out that is absolutely critical for any risk management methodology. Risk management has to be a living system! It is no good to do your risk analyses and then (as Bills says) simply put the documents in the file and forget about them. In the real world, your experience with the product will continue to grow: every hiccup in production and every customer complaint tell you something you didn't know the day before. And this means you have to pull your risk analyses out of the file, update them with the new information, and check to see how that changes your results. 

For example, maybe some customer finds a way to misuse the product that you never thought of (customers are good at this) and as a result it breaks. (Or, far worse, hurts someone.) Now you know that the product won't withstand that kind of misuse. That's new data, so you have to add it to your risk analysis as an input and then see what changes.

There's a lot more, of course, and if you work in the medical device industry I strongly recommend the article. But these points should apply across all industries, so I am grateful that Bills spelled them out.

           

Are management systems really systems?

This last week I've been reading about basic systems theory—specifically, Donella H. Meadows's primer, Thinking in Systems. And I've started to wonder about the management systems that we work with in the Quality business. Do these qualify as "systems" in Meadows's terminology? And if so, can systems theory teach us anything useful about how to design or implement a QMS?

The answer to the first question is an easy Yes. Meadows defines a "system" as "a set of things—people, cells, molecules, or whatever—interconnected in such a way that they produce their own pattern of behavior over time." Since a management system is "a set of policies, processes and procedures used by an organization to ensure that it can fulfill the tasks required to achieve its objectives," it obviously qualifies. But the second question is more interesting, and I don't have a final answer yet.

What I have seen is just how well her basic list of the features of systems maps to our everyday experience. In Chapter Three ("Why Systems Work So Well") she lists three important features.

Resilience

Systems generally have feedback loops that allow them to self-correct when they go too far out of alignment in any direction. This doesn't necessarily mean that systems are static—quite the contrary! But it does mean that they respond and adapt to changing circumstances. And of course management systems are designed to do this too. 

• Organizations set goals, and then evaluate whether those goals have been met; in case the goals were missed, the organization has to make a decision how to react, and this typically means analyzing what went wrong and then correcting the goal (or the system) based on the analysis. 
• Formal management systems generally require internal (and sometimes external) audits, in order to find places where the system has broken down or is not effective; then the responsible functions have to take corrective action to repair the situation. 
• And formal management systems require some kind of regular management review, which again looks for areas that have to be improved. All of these methods involve providing a kind of feedback to the organization that allows it to adapt to circumstances.

What's more, Meadows's discussion of resilience also picks up some of the dysfunctions we have discussed when talking about bureaucracy. She recognizes that bureaucracy is how large organizations communicate internally, but when it gets too large the communication becomes too slow. 

Large organizations of all kinds, from corporations to governments, lose their resilience simply because the feedback mechanisms by which they sense and respond to their environment have to travel through too many layers of delay and distortion. [p. 78]

That was exactly the point of discussion between Scotlyn and me last spring.

Self-Organization

Normally when we ask about an organization's management system, we mean the part that is defined in the documentation, the part we can audit. But every organization displays regular, stable behaviors that are not covered in the documentation. All of these are the result of self-organization on the part of the people in the system, and they have an enormous impact on the way work gets done. 

Some of these behaviors can actively subvert the purpose of the management system, such as when people game their metrics to ensure they are always green.

Sometimes the behaviors don't affect the management system directly for good or for ill, but they define the spirit or culture of the organization.

And of course sometimes the unplanned (self-organized) actions of the people on the floor can be an enormous help. The whole reason that our profession now uses Quality Circles is that they are a formalization of what must originally have been a self-organized discussion among line workers who were trying to solve some production problem. Today we say that Quality Circles are part of the formal, documented system; but it is not possible that they started that way. We all know that informal methods always precede formal ones.

Hierarchy

Clearly every management system assumes the existence of some kind of hierarchy in its organization. The ISO 9001 standard, for example, has multiple references to "top management." (See clauses 5.1, 5.2, 5.3, and 9.3.) One of the first artifacts we auditors ask an organization for is an org chart, so we can understand the hierarchy and the associated responsibilities. Even in cases where a group of people are working together without any hierarchy at all—at first—Meadows says you can watch one grow spontaneously (as a result of self-organization!) in order to reduce chaos and confusion by imposing some kind of organization. (The German sociologist Robert Michels once stated, "Who says organization, says oligarchy.")

What fascinates me is a remark Meadows makes about how hierarchies develop:

Hierarchies evolve from the lowest level up—from the pieces to the whole, from cell to organ to organism, from individual to team, from actual production to management of production. Early farmers decided to come together and form cities for self-protection and for making trade more efficient. Life started with single-cell bacteria, not with elephants. The original purpose of a hierarchy is always to help its originating subsystems do their jobs better. [p. 84. Emphasis added.]

This means the next time I do an audit, I should be able to ask the top management, "How do you help the people on the production line do their jobs better?" I want to remember that!

So management systems are indeed "systems" in the technical sense, and the common features of systems are things we all know from long experience working with management systems. I haven't decided yet whether there are other lessons to be derived from systems theory that might help us in the Quality business to do our jobs better. But I am continuing to think about it. If you know more than I do about this, please say so in the comments.

     

Lizard People and the Context of your Organization (My first podcast!)

Last week I sat down with Kyle Chambers of Texas Quality Assurance to record my first-ever podcast. We had a wide-ranging discussion: the original idea was that we would talk about Context of the Organization and how your COTO analysis can benefit you. But while we were at it, we also talked about how to design your QMS, why policy matters, how to avoid common pitfalls in auditing ... and how the world is secretly run by intergalactic space lizards!

Check it out.

You can find the podcast version here: #QualityMatters episode 147.

Or there's a version that also includes video, which you can find here: 

Leave me a comment to let me know your thoughts!

Context and scope

Last week I talked about how a good understanding of the context of your organization (COTO) can support your identification of business risks, because what business you are in affects what kinds of risks you face. But your COTO does more than that. A well-analyzed COTO can clarify the scope of your Quality Management System as well. 

Image by Mohamed Hassan from Pixabay

Some years ago, I audited a service organization that had a very special relationship with their one and only customer. This company provided services to a particular campus of a large, global organization that I'll call Octopus Enterprises. (That wasn't their real name.) But the relationship was close enough that my client's personnel were actually located on the Octopus campus, on a couple floors of one of the buildings. And while the annual contract spelled out that my client was responsible for this and that services specifically, everyone understood that Octopus could ask for something new tomorrow and my client would do it. There might be a discussion about the price, but there would be no discussion about whether to take the job. 

The first time I was there I asked how they defined their management system, and they handed me a Quality Manual which looked exactly like every other Quality Manual you've ever seen. And so I asked the General Manager:

"Why did you write this?"

He said, "I thought we were supposed to have a Quality Manual."

"OK, that was the old edition of the standard and you don't have to any more. But there's a bigger issue. 

"Look, this says you have procedures and follow them. But if Octopus tells you all to 'Jump,' you jump, procedures be hanged. If Octopus tells you all to wear bright green hats to work, you all wear bright green hats to work. It doesn't say that here. 

"I understand that legally you are a separate company. But if Octopus says to do something one day that violates your procedures, you'll do it. And then you risk having me or some other auditor write you up for not following your procedures. 

"For your own protection, you should have something in your management system that explains the relationship, and that says Octopus can override your normal procedures under such-and-such conditions."

I know an auditor is never supposed to consult, but it was important for them to understand this point.

Anyway, that's what I mean when I say that understanding your COTO can clarify the scope of your QMS. Understanding your COTO means you know who wants things from you (your interested parties) and what they want (their requirements and expectations). When you know those things, you also know how much influence these interested parties have over your decisions. If your organization disagrees with a requirement from some interested party, when can you say "No," and when do you have to say "Yes, right away"? If any interested party is important enough that they can affect or override your Quality System, they ipso facto affect your scope and you should make sure to say so. Carefully understanding your COTO is what tells you this.

          

Quality management in a learning organization

I mentioned before that the American Society for Quality held their World Conference on Quality and Improvement last month. As with any such conference, there were keynote speakers and then multiple breakout sessions to present special topics. One of the side-effects of this format was that different topics built on each other in unexpected ways. 

And so early on the second day I found myself wondering, We talk a lot about "learning organizations." If we want to create such an organization, how should we design the management system in order to support it? I don't claim to have an answer that is final or complete, but I started to collect some notes. Also, I hope you will join the discussion with ideas of your own. With luck, your ideas will be better than mine, and I will learn something.

I said this topic occurred to me early on the second day. The inputs from the first day, that started me thinking in this direction, included the following:

• Ugwem Eneyo, CEO of Shyft, discussing in her keynote address how technical development in Africa is leapfrogging over the stages familiar in Europe and North America to move in new and unexpected directions
• Charles Cox, on "QMS for data driven decision making and growth"
• Jane Keathley, on "Aligning organizational structure with Quality 4.0"
• Jeremiah Genest, on "Enabling the process owner to drive improvement"
• Heather McGowan opened the second day with a keynote on the rapid rate of change in the modern economy.  

And my initial thoughts, in no particular order, included the following.

A Quality Management System includes the following elements:

• analysis of the context of the organization
• policies, procedures, and records
• roles and responsibilities
• competence management
• tools for problem-solving
• internal audit
• management review

Regarding documentation and records, Cox made the point that, if you don't know what you need, all documentation systems look equally good. But in fact, different systems are good for different things. A learning organization needs the ability to create testable hypotheses and then record the results correlated with those tests, in order to evaluate the results for future planning. In addition, a learning organization has to get feedback as soon as possible; so whenever possible, long-lagging indicators should be replaced with short-lagging ones, and short-lagging indicators should be replaced with leading ones. It's not always clear how to define KPIs in this way; but whenever it's possible, it should be done.

If the organization truly is learning how to do things in new and better ways, roles and responsibilities are unlikely to remain fixed. Keathley described a networked organizational structure as one in which organizational charts are temporary, designed around particular initiatives; and such a structure sounds like it should be appropriate for a learning organization. There may be constraints imposed by the technology you have to deploy: the structure in a manufacturing plant is likely less fluid than that in a service organization, because using the machines correctly requires specific technical skills that not everyone shares. But again, so far as a networked structure is possible, it is probably the way to go.

In a learning organization, there is always the risk that a specific set of skills, or a specific job classification, might become obsolete—a risk that McGowan explained is inherent in the modern economy anyway. In this context, competence management means more than just training people to do a job. Rather, for each role the organization should identify the needed competence profiles, and then forecast what jobs with similar profiles are likely to be needed in the future. The people working in today's jobs should then be offered continual learning opportunities to expand the number of useful roles they can fill. 

By the way, it is critical that these opportunities be communicated in the right way. The message must never be, Your job is going away so you better learn something else instead. In the first place, that message is dispiriting; in the second place, it could be false. A "learning organization" is one that doesn't know the future yet; so the expectations for the future might not come to pass. A better message is, You are already good at A; we think that in the future we might need someone who knows how to do B, which is similar; and you already know us and our organization, so you would be a natural for this new role if it opens up. Would you like to learn more about it? Then even if it turns out that you don't replace role A with B, your organization will be stronger for investing in employee competence. And your employees will know that they get something out of it too.

In a learning organization, internal audits take on a new role because they are no longer simply tools for enforcement. Yes, of course an auditor has to check whether the defined procedures are being followed. But the auditor also has to check what results the organization is achieving. If the procedures are not strictly followed but the results are still good (like in my story half a year ago about the plating bath), this may be evidence that the organization can rewrite the procedures to make them looser or more generous, or to allow alternative ways to get the job done. If the procedures are followed perfectly but the results are poor, this too may be evidence that it's time to rewrite the procedures, because obviously they are no longer effective. Either way, the organization has to treat the audit as a fact-finding event, and then has to evaluate what the information really means.

Management review may change most of all. During management review, you check all your process metrics, to assess how things have gone in the last quarter or the last year. But in a learning organization, the point—oddly enough—isn't to meet your targets. The point is to learn how to do better. Targets in a learning organization are set at least partly as scientific experiments, to test hypotheses. If the numbers come back green, that shows the hypotheses were correct: but that's not necessarily a good thing. It depends where you want to go next. On the other hand, if the numbers come back red, that shows the hypotheses were incorrect: but again, that's not necessarily a bad thing. Even if your first guess turned out wrong, you now have more information than you had before. You have learned something. And you can do better next year. 

There's another element to learning, too, and management review is where it happens. Peter Drucker used to make the point that companies must regularly re-think what business they are really in, because the business landscape is always changing. Who are your real customers? What do those customers really want? These answers are never fixed, but too many companies treat them as immutable truths, laws of nature that never have to be reconsidered ... until one day the company goes broke because its old market ceased to exist while nobody was looking. It is exactly these questions to which a learning organization must learn and then relearn the answers, evaluating all the data from the whole Quality Management System in order to keep a step ahead of obsolescence. But to do this, top management has to use management review to force different questions than it asks in most companies. And it can never settle for easy answers.

All this is just a sketch, just a beginning. There is far more to work out. So now it's your turn. Leave me comments on this post to tell me where I'm wrong, and what I've missed. Figuring this out will be a collaborative effort, and I know I need your help.

What is more, our organizations need to know the answers so they can become better and more resilient. So please, give me your feedback.

                    

Where do you start?

"Congratulations, and welcome to WhizzBang Corporation! We're so glad you accepted our offer. It's good to have you aboard, and it's great to know you'll be taking over our Quality system. Now that you're hired, let me be completely candid. Things aren't where we want to see them. You'll find there are ... issues. I won't go into the details now. Better you look at it all with fresh eyes. But we know you're talented. Your previous experience is really impressive. So I'm sure you'll have everything shipshape in no time. Just remember, we're all counting on you."

Just what you wanted to hear, huh? You've just now taken over responsibility for a Quality system somewhere. Things are wrong and it's your job to set them right. But you don't actually know what's wrong, and it doesn't look like anybody's going to tell you. Besides, even if someone tells you where he thinks the problem is, how do you know that he's right? Maybe he doesn't see the whole picture. Or maybe the real problem is in his area, and he's trying to deflect your attention. What information can you trust, and what is just shadows in the fog?

Where do you start?

Audits. Start by doing internal system audits.

The thing is, unless you are prepared to tell everyone to scrap everything they are doing today and start over, you need information. But it has to be objective information. Confidential remarks that "Frankly, it's all Fred's fault" aren't good enough. A complete system audit won't catch every single problem, but it should highlight the areas of real concern and give you a good overall map that will let you plan your next steps.

What's more, you know you are going to get pushback when you try to correct things. People will tell you that you just don't understand—this is the way they've always done it, and it has always worked fine. But an audit gives you objective evidence. With that evidence, you can plainly show that the process is supposed to generate one kind of result but in fact it is generating another. It's not working. It has to be corrected.

Where you go next depends on what you find, of course. But start with a review of the whole system's status, backed up by objective evidence. Start with audits, and then go from there.

P.S.: I am assuming in all of this that the management system you work to actually provides for some kind of internal system audits. If your organization is governed by ISO 9001, ISO 14001, ISO 45001, or many others like them, it does. The same is true for most other management systems that I know of. In the unlikely event that you somehow find yourself in a situation where there is no standard provision for internal auditing, implement them anyway in an ad hoc way. If anybody asks, you can always claim that you're doing a gap analysis.    

      

How to work from home

I keep seeing articles that claim work-from-home is here to stay. But if that's true, we should figure out how to institutionalize it. We should figure out how to build it into our management systems.

When I google "work from home checklists," I see lots of them out there. So far the ones I've looked at seem to be about how to set up your space at home, how to keep your kids and your dog from interrupting you, how to keep yourself on-task instead of on YouTube, how not to spend all your time eating snack food ... things like that. And of course those are all important topics.

But so far I haven't seen any checklists that ask if you have actually coordinated your work-from-home with your business's systems and processes, or whether those systems and processes can continue to support you. I haven't seen any that ask you to think about the risks of working from home, even though I assume that both employees and supervisors have their heads full of risks that they worry about every day.

So I decided to give it a try.

On my Downloads page, you can find two checklists: 

• WFH_organizational-planning
• WFH_employee-readiness

The first checklist, as you might guess, is for an organization planning a work-from-home program. The intent is to cover all ways that such a program might affect a company's Quality Management System, so there are no unpleasant surprises where your external auditor thinks you sent everyone home without checking for risks, or for an impact on stakeholders.

The second checklist is for the individual employee. It's all very well for your company to be prepared to send you home to work, but are you prepared? Do you have a desk you can use? Do you have the equipment you need? Are you going to have to do something new for time-tracking? And so on.

Both these checklists sit at a very high level: you may need to attach other documents to spell out the details. But the details will differ for every organization. My goal is to cover all topics in breadth, but to leave the depth to you.

Also, both checklists are built around the ISO 9001 standard. That's not just a failure of imagination on my part, but a test. ISO 9001, after all, claims to model how any organization's management system ought to run. But that should mean that any model built on it is complete. In particular, if I write these work-from-home checklists based on the clauses of ISO 9001, I should have covered everything that has to be covered. If you find something important missing, then either I goofed or there is a hole in ISO 9001.

So download these checklists. Please. Look them over and try them out. 

And then send me your feedback. Tell me what's missing, or what's wrong, or what's awkward.

And if you find them useful, please tell me that too. Thanks!

           

Rethinking the case of universities

A couple of weeks ago, I discussed the question whether it makes any sense to certify a university to ISO 9001. More exactly, the original context of the discussion was the question whether ISO 9001 certification could replace academic accreditation. To that narrow question I finally answered No, but I left open the possibility that ISO 9001 might have some residual utility in organizing a university's activities.

Several people commented on the post (mostly through LinkedIn), but the most detailed response came from Rasha Alkabbanie (see her LinkedIn profile here), a Lecturer in the Faculty of Engineering of Tishk International University in Erbil, Iraq, who has studied – and published on! – the applicability of ISO 9001 to university operations. Besides sharing her expertise in an extensive comment, Ms. Alkabbanie linked to an article she published in June 2020, where she compared ISO 9001:2015 with other guidelines for institutions of higher learning. In particular, she focused her comparison on a set of international standards published as the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG), (Brussels, Belgium, 2015), or "ESG 2015" for short.

The argument in Ms. Alkabbanie's paper, "ESG 2015 vs. ISO 9001:2015 Regarding Stakeholders," turns implicitly on a subtle point which I failed to consider in my own blog post. This point is that a university, like any other organization (and particularly any other large-scale organization) has to have some kind of Quality Management System underlying its operations. 

• This system does not necessarily have to be called a QMS, though clearly sometimes it is; ESG 2015, for example, makes extensive use of "quality assurance" terminology. 
• It is even possible (I suppose) that some institution might not recognize that they have such a system in place, thinking of it simply as "the way we've always done things" without taking a step back to see how all of "the things we do" fit together as a coherent system when you see the picture whole.
• But what is not possible is that an underlying QMS actually fails to exist. A QMS, after all, is just the basic organizational framework of roles and responsibilities, and of activities and controls, that allows the institution to get things done. If a university – or any organization – can keep its doors open, then one way or another it has some kind of (formal or informal) QMS.

And as soon as you recognize that a university has to have some kind of QMS, you see how ISO 9001 could apply (subject as always to my qualifications in last week's post). But please note: this does not mean that ISO 9001 can take the place of accreditation. It can never do that, because accreditation imposes specific content-based requirements related to subject-matter expertise; or to professional qualification; or to expectations from regional, national, or international bodies. These content-based requirements can never be subsumed under a generic management system standard, in the same way that every industry has its own specialized requirements not shared by others. We all know that restaurants, and electronics manufacturers, and trucking companies can all implement ISO 9001, but each one has industry-specific requirements that the others don't. In the field of education, accreditation requirements play the same role.

But in order to meet these requirements, the university has to have some kind of system for meeting requirements in general; and that system can be judged by an external standard. Pick whichever standard you like: for example, you might choose ESG 2015, or you might choose ISO 9001. And here we come to the heart of Ms. Alkabbanie's research: comparing these two. Her conclusion is that ISO 9001 is a more demanding standard than ESG 2015, because it is more comprehensive. In particular, her analysis shows that ESG 2015 is somehow a subset of ISO 9001: every requirement in ESG 2015 is matched by a requirement in ISO 9001, but the reverse is not true. There are requirements in ISO 9001 that ESG 2015 never dreams of.

Perhaps I should add that there are other applicable standards you could choose from, besides these two. ISO 21001, for example, is specifically a standard for "Management systems for educational organizations." It resembles ISO 9001 very closely, adding a number of topics that are specific to educational institutions: for example, admissions criteria, grading of student work, and policies for special-needs students. But then in the supplemental material at the end, in Annex F, ISO 21001:2018 provides a comparison between its own requirements and those of yet another standard, in this case the European Quality Assurance Framework for Vocational Education and Training (EQAVET). (As a confirmation that every educational institution has to have some kind of QMS, the EQAVET web page – just like ESG 2015 – makes heavy use of "quality assurance" terminology. It also deploys a version of the Plan-Do-Check-Act cycle.)

So that's the relevance of ISO 9001 to universities: as an enabler of accreditation, not a replacement or substitute for it. And I am very grateful to Ms. Alkabbanie for her detailed comments, and for her willingness to share the results of her research, all of which helped me better understand this point.