"People Before Process"

A few weeks ago, I had the good luck to attend a webinar called "People Before Process." This webinar was a real treat. It was clear, engaging, and insightful. And it touched on themes that I have discussed before about the role that defined processes play in achieving Quality — in getting what you want.

Before I forget, here are a few particulars. The talk was sponsored by the ASQ's Quality Management Division, and speaker was Jeff Griffiths ("About" page, LinkedIn). If you are a member of ASQ, you can access the video here. But he discusses some of the same concepts in less detail here (no need for an ASQ membership) and you can find him in a number of video conversations here.

Griffiths's fundamental point throughout these talks is that, if you want to get results, there is no substitute for having the right people to get you those results. Starting from that foundation, he then discusses various ways that an organization can plant, nourish, and grow the needed competencies in their people. In the webinar I joined he introduced the Dreyfus model to distinguish multiple levels of skill acquisition, and described client quality problems that his firm had helped resolve specifically through enhancing worker competency rather than by introducing new procedures.

To make the point that people are more important than process, Griffiths proposed an interesting thought experiment. Suppose, he said, you have a table like the one below, and that your organization can fall in one of the four quadrants depending whether your people are strong or weak and whether your processes are strong or weak.

Of course anyone who has a choice wants to be in quadrant II, with strong people and strong processes. Likewise we can all agree that our last choice is to be in quadrant IV. That part is easy. But what if we have to choose between quadrant I and quadrant III? Griffiths argues that we are far better off in quadrant III, because if the people in the organization are fundamentally strong and competent — but they have been saddled with processes that are weak or badly-designed — the people will change the processes into something that works better, and thus will pull the organization in the direction of quadrant II. But if you have weak people — poorly trained, uncaring, or actively disengaged — the best process in the world can't overcome them.

Quadrant III is better because it is temporary: it is always pulling towards II.

Is it true? I think so. Even when an organization is not in control of their own processes*, they can make improvements at a daily level by interpreting the rules so they support the work, applying and enforcing them in ways that are helpful and productive. There is no single instruction for how to do this that fits all cases, no one-size-fits-all formula. Each situation has to be evaluated on its own. But in my experience it is possible.  

When I look at the table, I see something else too, something Griffiths never says explicitly. I bet that quadrant I is always pulling towards quadrant IV. Think about it. Suppose you have an organization with an excellent set of written processes, but where the people are poorly trained for the work and don't understand the processes — or don't care. What happens? That's easy: Scott Adams has made an entire career writing about it in Dilbert. Look at all the jokes made at the expense of ISO 9001: the standard itself is more or less a body of formalized common-sense, but when it is badly implemented or badly-applied it becomes a punchline. And so, bit by bit, processes which were once helpful and robust are misused and misapplied; enforcement is either too strict or too loose (or veers unpredictably between one and the other); the processes thus become obstructions rather than enablers; and the organization drifts from I to IV.

If people are so much more important than process, why do I write about process? The simplest answer is that I write what I know; further, I never said process was irrelevant. Obviously your business processes and the structure of your QMS still make a significant difference to your outcomes. But the overriding theme of this blog is that any QMS has to be applied pragmatically; this means that the system itself can never solve all your problems. The centrality of your people is one huge reason why not. 

If you find yourself wondering what to do about that fact and where to turn next, check out Griffiths's organization and blog. You'll find some advice there. 

__________

* This can happen in, say, a global company that requires all units to follow the same processes even if they do different work.

Do audits really add value? Part 3 of 3

In the last couple of weeks, I've discussed the question whether external, third-party audit results are reliable. On the one hand, I've given reasons it is fair to be suspicious of them; on the other, I've had experiences where they have proven uncannily perceptive. What's the middle ground, the synthesis of these two conflicting positions? Last week I tipped my hand by saying that "yes, we can trust our audit results, provided that we understand clearly what the job of an external audit really is and don't expect it to do something else instead." In what follows, let me try to spell out what that job is.

What an external audit is not

In the first place, an external audit is not a complete health-check. After all, it is a commonplace that auditing is a sampling operation. Third-party auditors routinely remind clients of this during their Opening or Closing Meetings. And in an earlier post I mentioned an audit instructor who said clearly, Any time you do an audit, there will be minor nonconformities that you will miss. Even if your organization passes, that doesn't guarantee that everything is perfect. Notice that for this reason, it is not necessary for an auditor to make the experience painful for the client, because he's not even trying to catch everything. So if someone (like the fellow I quoted two weeks ago) says that auditors are taking a "kinder and gentler" approach than they did back in the early 1990's, that doesn't have to be a problem.

That's what an external audit does not do. Now what does it do?

Enforcement

The first thing an audit does is to support the enforcement of the organization's Quality Management System. Every QMS involves imposing a set of rules on the organization; and no matter how engaged the employees are, there will always be someone who thinks that this particular rule shouldn't apply to him. And maybe for a while he gets away with it: management's attention is somewhere else, and his colleagues don't feel like leaning on him. But sooner or later somebody schedules an audit. And then the message — from management and colleagues alike — suddenly becomes the same: Dude, even if you think the rule is dumb you have got to comply with it or else the auditor will write us up.  And that message is often convincing even when no other message has worked.

Fresh eyes

Sometimes there is something wrong in your system which you know is wrong, but you walk past it every single day and after a while you stop seeing it. I had something like this happen to me. The local organization I worked in had a procedure for processing 8D problem reports. It was based on a global procedure that covered our whole division worldwide, but there were local adaptations for one reason and another. Anyway, the global procedure changed, which meant that we (that means I) had to change the local one to match it. The adjustment was straightforward; I knew exactly what I had to do. So I put it on my to-do list. This was six months before our next surveillance audit.

You know what happened next. One thing and another interrupted me before I could work on it immediately, and then it slid far enough down the list that I didn't see it often. Occasionally I would notice it and remember, Oh right, I still have to fix that. But about that time another problem would cross my desk and I'd forget again.

And then our auditor showed up. As we reviewed the corrective action system, he asked to see our 8D procedure. I gave it to him, and he read to about page 2 where he suddenly asked, It says here you process 8Ds like this. Is that true? And then I remembered, Oops! Not any more we don't. I was going to fix that, wasn't I? Of course he wrote a nonconformity, and to answer it I finally updated the document correctly. It doesn't seem like a big issue in the grand scheme of things, but if he hadn't written that finding I might never have remembered to do it. And as I discussed last summer, it actually does matter that your procedure documents be accurate. 

System integrity

There is at least one more job that an external audit does reliably. It guarantees the overall integrity of the system. To explain what I mean, let me tell another story.

Years ago, I worked in a place that was struggling to implement a disciplined QMS. We had gotten ISO 9001 certification, but keeping things at a sustainable level was a challenge. It seemed like every year I was writing Major Nonconformities in our internal audits.

So after one of our external surveillance audits, the General Manager took a few minutes out of his next staff meeting to complain that the audit process was useless.

Me: What do you mean "useless"?

General Manager: Well that guy spent a few days here, he seemed to talk to everyone, but then he gave us a clean bill of health! What's wrong with him? Didn't he see that we had seven Major Nonconformities in our internal audits? How could he say that our system is working OK?

I wasn't sure how to answer, so after the meeting I forwarded that question to the auditor (whose contact information I had kept). And he answered:

Auditor: Yes, I saw those seven Majors. But you found them, didn't you? They were all clearly stated in the internal audit report; and when we checked the action plans, the root-cause analyses looked reasonable and the corrective measures were on-schedule. The system was working exactly the way it's supposed to work.  

Then he went on.

Auditor: Look, if you want me to come out there, photocopy your internal audit results, sign my name to them, and then spend the rest of the week in a bar — and get paid for it — I can do that. But that's not going to give you a lot of value. So it's more important to me to make sure that your overall system is hanging together and functioning the way it should. Of course you're going to have problems or hit bumps in the road. That's normal. The important part is how you react to those problems, and right now you guys are doing fine.

And that's what I mean by guaranteeing the overall integrity of the system. This is why an external auditor doesn't have to find every little thing the organization is doing wrong: because if the system is working correctly, the organization will find those problems themselves. Therefore the one critical thing that the external auditor has to ensure is that the system itself is working.

This point relates also to our earlier discussion of the difference between Minors and Majors. Two weeks ago, when I listed reasons to be suspicious of audit results, most of those reasons applied to Minors. Didn't they?

• If auditors used to strain at gnats and no longer do, that has to mean that they used to write a lot of Minors and no longer do, because writing Majors has always been the exception. 
• More to the point, think about the external audit that started this whole train of thought, where the auditor asked a few simple questions and then wrapped up the audit. What made that possible was that the overall system was functioning just fine — and in that office, by that time, it was. Yes, if he had been more focused he could have found a few Minors for us to chase after. But fundamentally that's not what we needed from him. We had internal audits for that — and customer complaints, and nonconforming material reports, and the whole armamentarium of Quality Management tools. What we needed from him was assurance that the system was intact, and it was. 
• And while experts certainly disagree, I would argue that they are a lot more likely to disagree over Minors than over Majors because Minors are one-off failures. They are almost incidental. And therefore there is a lot more room for personal, subjective judgement to come into play. Majors, on the contrary, are by definition failures that endanger the system. My old instructor might have been exaggerating when he said that "if the organization has Majors, you will know it by the time you reach the Receptionist's desk!" But it is pretty hard to mistake a system breakdown for a one-off failure, or vice versa.   

From this point of view, the most important job of the external auditor is to find and report Majors, if there are any. Minors are lagniappe. If the external auditor happens to find them, of course he reports them; but if he doesn't, somebody else will. On the other hand if the system has broken down, that "somebody else" might never come along. So the external auditor has to report on Majors. 

And for that reason, as long as we remember the difference between what external audits must do and what they cannot pretend to do, we can continue our audit programs with a good conscience.

         

Do audits really add value? Part 2 of 3

Last week I asked whether we can ever trust the third-party audit process, and suggested two (or maybe three) reasons we might not: registrars go easy on us because they don't want to lose clients, experts disagree, and (although this last point is a very individual matter) there are a few external auditors who are kind of goofy. (We've all met one somewhere.)

On the other hand, I've also had experiences that run smack in the other direction. Let me describe two.

One time I worked with an external auditor who I thought was going to end up in the "goofy" category. She was a chatty little old lady, who always started off her interviews by talking about her vacations or her grandkids, and who invariably wound up her interviews ahead of schedule. Then as soon as the auditee had left the room she'd ask me to step outside with her so she could have a cigarette. All through the audit, I had people comment to me quietly that they were amazed how smoothly it was going. And then at the end she wrote us three nonconformities which exactly nailed the three places we were having the most trouble. Her style was so relaxed that it put everyone off-guard, but her questions probed deeply — and got there fast.

There's also a phenomenon that I have experienced when I do internal audits, and that I have seen play out in external audits as well. I have sometimes jokingly called it a Special Providence for ISO Auditors. It works like this: 

• The client has a drawer full of 100 files. 
• Ninety-eight of those files are perfect. Two are wrong. 
• You, the auditor, close your eyes and randomly pull three files out to check.
• One of the ones you pull out will be wrong.

It's uncanny how often this works. I've had it happen to me when I do internal audits, and I've watched external auditors do the exact same thing. Those auditors wrote us up, too.

So where does this leave us? Last week I talked about reasons not to trust audit results. But this week I've discussed two reasons we can trust them: trained auditors can be amazingly perceptive, and problems or errors seem to jump out in front of them. Which is it?

My answer is that yes, we can trust our audit results, provided that we understand clearly what the job of an external audit really is and don't expect it to do something else instead. I'll talk about the real job of an audit in next week's installment.

     

Do audits really add value? Part 1 of 3

Auditing is a fundamental part of any management system. In the four-step cycle of "Plan-Do-Check-Act," auditing is the core of step three, "Check." Without audits we would never know if our systems were working.

But there are internal audits and external audits. And is the information we get from external audits actually reliable? Is it useful? Or are there institutional or organizational factors that skew or corrupt the auditing process?

I think the real picture is not a simple one, so I'll break up my answer into three parts, all based on my own personal experience. In the first post — this one — I'll give reasons to suspect that maybe external audits don't really add value. In the second post, I'll backtrack and give reasons that maybe they do add value after all. And in the third post I'll try to find a middle ground that does justice to both opinions.

You may be able to guess where I'll end up before I get there, or you may have opinions of your own. Either way, feedback is always welcome. Please feel free to comment.

My thinking along these lines all started after a singularly-unimpressive third-party audit. The auditor walked around, asked a couple of aimless questions to which we gave answers that he apparently found acceptable, and wound up. All done. 

Afterwards I sat around talking with the site management, and we tried to understand what had happened. Partly it was just that this particular guy was a little goofy, and we've all met auditors for whom we could say the same. (Christopher Paris even jokes about it in his auditor cartoons.) But the General Manager remarked that our experience that day seemed, in his mind, about par for the course. He said that all the auditors he could remember seemed to have been more-or-less colorful individuals, but he couldn't think of any findings that made much of a difference to the organization. So what's the point, he asked. Is there any real value that we get out of interrupting operations for a day or two while we host these guys? Do their reports actually help us build a better organization? Or is this all just a song-and-dance we have to go through as part of the price for getting our certificate? Because we really do need the certificate. But is that all we're getting?

In reply, I explained something that another third-party auditor had described for me several years ago. When ISO 9001 was first introduced, so this man had told me, auditors were very strict. Getting certification was a big deal. But over time there were more and more registrars available, and they were in competition with each other. This meant that if a company was refused certification by, say, BVC or BSI, they could always call up DNV or DQS or Perry-Johnson and try again. The risk of losing repeat business pushed the registrars to grade more and more leniently, and to market themselves as “partners” in improving your management systems. They wrote fewer nonconformities and more “opportunities for improvement” … suggestions for things you might want to consider. 

The General Manager listened to all of this politely. Then, bless his heart, he jumped in with exactly the right question: You know who else behaved like that? The bond-rating agencies, back in 2008. And see how well that turned out for everybody!

Even when auditors aren't deliberately throwing slow softball pitches, though, there's another risk to the reliability of any audit report. Experts disagree. Often they disagree a lot.* If you have ever performed an audit and then discussed it with another trained auditor, you know that no two auditors will choose to include exactly the same data points; and even if they can agree on a particular finding, it's not infrequent that they'll rate it at different levels of severity. We always ask for "objective evidence" to justify any finding, and rightly so. But what we then do with that objective evidence depends more than we like to admit on subjective assessments and personal expertise.

So if audit reports are, in the end, based (at least partly) on the auditor's personal, subjective opinions — and if auditors are under institutional pressure not to be terribly hard on their clients — this brings us back to the original question: Do audits really add any value? And if you look at it from one point of view, you wouldn't be crazy if you suspected that the answer might be No.

But don't touch that dial. I'll come back next week to look at the question from another point of view.

 

__________

* You can find this topic discussed at great length in, for example, the medical field, where it would seem to be a matter of life-or-death to get the answers right. See for example this article from 2008, this one and this one from 2010 (both based on the same book), or this one from 2020. This extreme variability is one of the reasons that medicine relies more and more often on objective checklists instead of personal expertise, as described here.