Is that really a Nonconformity?

When you carry out a Management System audit, one of the tasks is to decide how to rate your findings. You stumble over something that's just not right, but is it really a Nonconformity, or is it something milder — say, an Opportunity for Improvement? If it's a Nonconformity, is it a Major or a Minor?

You'd think it would be easy to decide these things, and of course sometimes it is. Out on the production floor you find someone working to revision A of a certain procedure, but back in the office you already confirmed the current revision is B: obviously this is some kind of nonconformity against Document Control. What's interesting is that sometimes it's not so obvious.

This week I will talk about distinguishing Minors from Opportunities. Next week I will talk about distinguishing Minors from Majors.

The audit from Hell

Once upon a time, I was co-auditor on an internal audit of a manufacturing plant; the Lead Auditor (whom I'll call "Jackie," which was not her name!) was a colleague brought over from Europe. Jackie and I got on very well at a personal level, but when we compared notes at the end of the first day I was stunned: I had audited these departments over here and came back with something like four or five nonconformities, while she had audited those departments over there and came back with over a dozen. The next day, it was the same story. In the end (it was a four-day audit) we combined some of the findings to make the list shorter, but still we reported something like twenty-five Minors and one Major. 

Was the plant really in such bad shape? No, not really. They were sloppy about their paperwork and some other administrative topics. I would have been happier if they had taken those things more seriously. But none of this casual attitude found its way into the products. When you read the long list of findings in our report, none of them jeopardized customer satisfaction in any direct way. 

So why the ghastly report?

1. The plant had recently been acquired by a new owner, who had a whole set of their own procedures. (The new owner was headquartered in Europe, which is why Jackie was flown out to do the audit.)
2. Jackie was the Lead Auditor, which gave her the final say over what went into the report and how we should rate it.
3. And Jackie's perspective — every time she found anything out of alignment with the way the new owner did things — was to insist, "But the procedure clearly says …."

Sometimes that's the right way to audit, but this wasn't one of those times. Nobody in that plant learned anything useful by being forced to slog through a root-cause analysis for why the corporate document templates hadn't been rolled out yet, or why the plant hadn't implemented the standard corporate labeling conventions in the warehouse, or why the supplier audit results weren't posted on a website so they'd be easy to find. All they learned was to hate internal auditors, which didn't make my work any easier in the years to come.

And it was all so unnecessary. Of course the errors had to be written up. But there has to be room to apply a measure of judgement or discretion in the writing. The trick is to find rationally defensible criteria for that judgement or discretion, so that you don't just assign ratings based on how you feel that day. In the months following that audit, I chewed over this problem a lot. And I think I have an answer.

Minor Nonconformity or Opportunity for Improvement?

The key question is this: How mature is the organization you are auditing? The purpose of the audit is to drive continual improvement. That means the organization has to learn something from your findings, something they can use to get better. A truly mature organization can learn from even the most trivial findings. So if you are auditing a mature organization you can write up any mismatch between the procedures and what you observe in reality, confident that your finding will lead to improvement. If you are auditing an immature organization, though, you have to pick your battles. Here's how.  

Let's say you are auditing an organization that still has some maturing to do. During the audit, you identify several places where the procedures say one thing, and the auditees or operators do something different. Interrogate each situation as follows:

1. Check the procedure that your auditee failed to follow. Remember that nobody writes a procedure for no reason: every step in a procedure was written there in order to keep something from going wrong. Focus on the steps that your auditee failed to follow, and ask yourself: What is the risk that these steps were written to avoid?
2. Now check back with the auditee. Explain that the procedure was written to keep this or that bad thing from happening. Then ask: Do you have some other way to prevent these risks? Can you show me that there's no chance of these bad things happening?
3. If the answer is Yes, then from the perspective of the end result the auditee actually complies with the procedure because he has succeeded in eliminating the risk that the procedure was written to eliminate. 
• In this case, write an Opportunity for Improvement as follows: "Procedure 123 says to do … in order to avoid the risk of …. In fact the operator avoids that risk in another way, by doing …. Consider reviewing the procedure to determine whether it can be rewritten to match how the operator really works, or whether the operator should change his work to bring it in line with the procedure."
4. On the other hand if the answer is No, you have to write a Nonconformity. But at that point the discussion isn't about "compliance with the procedure"; it's about the live, unaddressed risk that could jeopardize operations. 
• In this case your message to the organization should be, "You folks are playing with matches, and it's a lucky thing you haven't burnt your fingers yet. Fix this before your luck runs out." You don't even have to mention "compliance with the procedure"; and the organization will be grateful that you identified a risk, instead of resentful that they have to fill out a bunch of audit paperwork.

For the immature or semi-mature organization, redirecting the focus like this is a big help.