Do you audit Controlling or Finance?

Last week I asked whether you rule Human Resources in-scope or out-of-scope for your internal audit plan. Today I want to ask the same question about the Controlling or Finance department. My answer here is a lot shorter than my answer about HR, and somewhat more tentative. In the end I still think it can be a good idea; but if you need to trim somewhere because of limited auditing capacity, this is a place you can safely trim.

The point is that ISO 9001 is not a financial standard, and – normally – quality auditors aren't trained as accountants, much less as financial auditors. So we are not likely to catch even arithmetical errors, much less any serious financial misdeeds (fraud, embezzlement, or the like). And I seriously hope that your organization's Controlling Department is already audited regularly by a reputable auditing firm, preferably while the department manager is out of the office on vacation. Anyway, those aren't the kinds of issues that we quality auditors are qualified to catch, and we shouldn't try.

What we are qualified to discuss is the effectiveness of overall business processes, and of the management system's process map. How do the different functions in the organization support each other? And the place that Controlling is most likely to enter this cycle is by supporting top management in the business planning process. 

Top management is responsible for defining the business's strategy, and then for steering and making adjustments as results become visible in the real world. Most often "results" means financial results, and Controlling is where they have to turn to request that information. So what exactly is the process by which Controlling feeds information into the business planning process? What information is collected? Why is that the right information to look at, and not something else? Also, how well does the whole planning process work? Does this information from Controlling actually tell top management what they need to know? Are there any ways the handoff could be improved, or the quality of the data itself?

If you don't have enough time in the audit plan – or enough trained auditors – you can probably afford to skip Controlling as a separate audit, and pick these topics up when you audit top management. In case there are any audit trails that lead you into Controlling you can follow them then, and otherwise it's likely that the most you will find are opportunities for future improvement. But if you can afford the audit capacity, you can learn a lot about the organization's overall management system by giving Controlling a short visit. That's why I still think it can be a good idea to rule them in-scope.

    

Do you audit Human Resources?

When you plan out your internal audit program for the year, is the Human Resources department in-scope?

It sounds like a simple question, but it's not. Or at any rate I've found that auditors I respect approach the question very differently from the way I do.

When I first started planning audits, it seemed easy: we were going to audit every department in the company, HR was one of those departments, ergo we audited HR. Simple enough. 

Then one day I was working with a colleague from another division – a fine auditor with decades of experience in the Quality business – who asked me Why? in genuine bafflement. 

His argument was that ISO 9001 is all about customer satisfaction. The whole point of a Quality Management System in the first place is so that you consistently deliver the goods and services that your customers require. So who cares what your hiring process is, or how you do annual reviews? None of those things affect the customer, so they should be out of scope for the audit program.

He did make an exception in case HR was in charge of the company training program. Training shows up under clause 7.2 ("Competence"); so if HR is in charge of training records (and sometimes they are) he was prepared to visit them so he could audit training records and ask about the process for ensuring that all training is up to date. 

But even when you look at the other clauses that discuss how the organization hires or treats its people – see, for example, 7.1.2 ("People") or the Note under 7.1.4 ("Environment for the operation of processes") where there is discussion of social and psychological factors in the work environment – HR's role is usually to do what they are told. The HR Manager isn't going to tell the Operations Manager that he needs another inspector on Line 3. Rather, it's the Operations Manager who determines that an extra inspector is "necessary for the … control of … [the] processes" on Line 3 and who then asks HR to go hire one. So if you want to check how that determination was made (because you are auditing 7.1.2), go ask the Operations Manager and not HR. As for social and psychological factors in the work environment, … when was the last time you saw HR tell Engineering that the product development deadlines have to be more generous in order to avoid burnout in the design team? Well, then.

It's a good argument, and I had to think about it for a while. In the end I think I still disagree, but I've had to build an argument of my own. Here is how I see it now: let me know in the Comments which of us you find more persuasive.

My first point is that ISO 9001 is no longer just about customer satisfaction. It's now about the entire functioning of the organization. When ISO 9000:2015 defines quality* as the "degree to which a set of inherent characteristics of an object fulfils requirements," those requirements can come from any interested party.** Interested parties and their needs are determined way back in clause 4.2, and the list of interested parties should certainly include the employees themselves (among others).

Now, consider an absurd example. Suppose the organization offers its employees health insurance, and there are three plans to choose from: call them Gold, Silver, and Bronze. And suppose further that, no matter how the employees fill out their paperwork, the organization's HR department always signs them up for Bronze!

Clearly in a case like that something has gone wrong in the HR process. It doesn't affect the quality of the goods and services that are delivered to the customer, but it seriously affects the quality of the services provided to the employees. So if I were the auditor I'd want to write a nonconformity against some clause – probably 8.5.1 ("Control of production and service provision"), because I would argue that something was badly wrong with the controls on (internal) service provision.

Naturally a breakdown like this is unlikely. But the point is that it is possible, and that the organization's employees are legitimate interested parties who deserve to have their needs and expectations considered as the organization defines its business processes. Or alternatively, if the organization has deliberately chosen not to treat the employees as interested parties, there should be some documented discussion of that decision in the overall planning for the QMS. And that documented decision itself should then be available to the auditor.

So I have persuaded myself that it is still fair and reasonable to rule HR in-scope for the internal audit program. But have I persuaded you? Leave me some remarks in the Comments to let me know.      

__________

* in entry 3.6.2

** see entry 3.6.4, Note 4

    

What's the difference between PDCA and a Management System?

Last week somebody posted a survey on LinkedIn that asked whether "PDCA (-Plan-Do-Check-Act-) Approach can be referred [to] as [a] Management System." You can find it here. The last time I checked (a few hours before it closed) it had collected 381 votes, of which 78% said Yes and 19% said No.

But this is not a matter of opinion. It's a simple matter of fact, and the correct answer is No. What this means is that nearly 300 people who felt confident answering an Internet poll about management systems actually don't understand what a management system is. How is this possible?

To be clear, I recognize that this is a specialized area. And it's not like I think that if I know something, then everyone should know it. There are lots of topics I know nothing about. But neither do I answer Internet polls about quantum mechanics, or cellular biology, or derivative-pricing in hedge-funds.

I can't answer the question why four-fifths of the people who thought they knew the answer got it wrong. But at least I can explain why the other answer is right.

A management system is like a Constitution: it provides the framework for an organization's rules, and it defines roles and responsibilities. More exactly, a management system defines the processes that you use to do the work of your organization, together with all their interactions and controls. You define the interactions because nobody works in a vacuum: your organization's processes must somehow interrelate or interlock. And you define controls -- of some kind! -- to keep things from going off the rails. Defined roles and responsibilities can be (among other things) a kind of process control; KPIs are another kind. These are all the topics we've been discussing here over the last couple of months.

One way a management system differs from a Constitution is that it is more complete. The Constitution of the United States, for example, tells us how laws are made and enforced but doesn't tell us what the laws themselves actually say. A management system includes not only the framework of processes and authorities, but all the individual procedures and work instructions as well. Every wall poster reminding you to be careful of the risks from your machinery, every company-sponsored training class, and every calendar entry telling you it's time to recalibrate your gauges -- all these are parts of the management system too. 

So if you think of a management system as like a Constitution plus all the subsequent laws and stuff … you won't be too far wrong. Obviously most organizations are a lot smaller than most countries, so their management systems are far smaller and simpler than any country's body of legislation. But if you can think of your company as kind of like a small country, the idea is similar.

PDCA, by contrast, is just a generic model for how to do things -- pretty much anything, in fact. The letters stand for Plan-Do-Check-Act, and you can think of them as a simplified summary of the scientific method. Whatever it is you want to do, start by making a Plan based on what you already know (or think you know); then go Do it; then Check to see if you got the results you expected to get; and finally reAct based on whatever you learned in the "Check" phase -- either by revising your assumptions, updating your procedures, or concluding that everything is perfect and steaming full speed ahead. (Notice, by the way, that the PDCA model is truly generic, while a management system is specific to this or that organization in particular.)

When I say you can use PDCA for anything, I really do mean almost anything. One way or another, most of us use the PDCA model intuitively without thinking about it. Giving it a name just helps formalize the steps so that we can be systematic and intentional about them.

For example, let's say you want to camp out in the woods over summer vacation, and you decide to manage the trip according to the PDCA model:

• During the Plan phase you choose the campsite, decide how many nights to stay, and plan what to take.
• During the Do phase you buy your groceries, load the car, and go camping.
• The Check phase is when you talk about it on the drive home. On the one hand, the mosquitos were a real nuisance. On the other hand, camping next to the lake was great because you could go swimming every morning. And you all agree that next time you've just got to bring a lot more marshmallows to roast over the campfire!
• Finally, during the Act phase, you make some notes of the things you learned this year that you don't want to forget (bug spray, lake, marshmallows) and you paste them on next year's calendar where you will see them next summer.

So of course you can also use the PDCA model when you are implementing and maintaining a management system, in exactly the same way.  

• During the Plan phase you set up all your processes, document them, and appoint people to their respective roles and responsibilities. 
• During the Do phase you go do the regular daily work of your organization. 
• During the Check phase you inspect the results: this might mean looking at your KPIs, or at the results from internal process audits, or something else of the kind. And of course part of inspecting the results is comparing them with your targets. Are the KPIs red or green? Did the audit reveal nonconformities, or not?
• During the Act phase, you make management decisions based on the results you found during the Check phase. If all your KPIs are red, is something wrong with your processes? (Or conversely, if they are all green have you been setting the targets too low?) If the same nonconformities keep showing up in audit after audit, is there some systemic problem that you have yet to find and resolve?
• Then based on the management decisions you made in the Act phase, it's time to Plan for the next period (e.g., for the next year). What KPIs will you measure in this year? Where will you set the target that divides green from red? Also maybe you have had to redefine this or that procedure, based on the systemic problems you uncovered above. All of this goes into the Plan for the next year.
• And then it's time for the next Do phase, once again.
• And so on, over and over.

It's a good approach. It's systematic, and if you apply it right you should learn from your mistakes over time. If you are regularly learning from your mistakes, you should be making the organization better and better. So by all means apply the PDCA model wherever you think it can help you.

But it's just a model, or a method. It can never be a management system by itself.

Have I made the distinction clear? Leave me comments one way or the other.

  

Who needs a Quality Policy?

Most corporate Quality Policies are worthless, and this is one area where the current ISO 9001 standard just makes things worse. But it shouldn’t have to be like this.

The whole point of setting a policy is that it gives you a framework for making decisions, so you don’t have to improvise every case from scratch. Let’s say you sell clothing, and one day a customer comes in to return a sweater. The sweater is made from some expensive fabric and has a label that clearly says “DRY CLEAN ONLY”; but the customer ignored the label, washed it with the rest of his laundry in an ordinary washing machine, and ruined it. So now he wants his money back. If you have a Returns Policy, then your sales associates know what to do: accept the return because keeping customer goodwill is worth putting up with a certain amount of foolishness; or tell the customer he's out of luck because he caused his own problem. No sales associate should have to figure out that answer in real time while the customer is standing there shouting. 

But what kind of decisions are you trying to support with a generalized Quality Policy? Does any employee ever tell himself, “I just can’t decide whether to do a good job or not; so I sure am glad there’s a Policy saying I should”? I didn’t think so.

What’s more, if you don’t know what problem you are trying to solve, you’re not going to solve anything very well. This is why most corporate Quality Policies sound like lukewarm vanilla pudding: they are not written to meet any real need, so they end up as meaningless blather.

This, in turn, enables a well-known auditor trick. Clause 5.2.2 (b) of ISO 9001:2015 requires that “The quality policy shall be communicated, understood and applied within the organization.” So if the audit is going slowly and the auditor wants to pad his list of nonconformities,* he can pull someone off the floor and ask, “What’s the Quality Policy? Explain it to me.” The employee probably went through a training session that covered this back when he was hired, but nobody in the company has ever mentioned it since and the employee has forgotten it. Even if there are posters on the wall spelling out the Policy, the employee can’t explain it because it’s lukewarm vanilla pudding. So the auditor gets to write a nonconformity almost for free. After the audit is done this finding will be assigned to senior management for corrective action; senior management will make everyone go to another training session on the Quality Policy; and a month later everyone will have forgotten it all again. The organization will have gone through lots of activity and will have gotten no meaningful result out of it.

Now, it’s possible to write a memorable Quality Policy. The problem is that ISO 9001 won’t let you. Clauses 5.2.1 (c) and (d) specifically require that your Quality Policy “includes a commitment to satisfy applicable requirements,” and “includes a commitment to continual improvement of the quality management system.” Christopher Paris, in his delightful book Surviving ISO 9001:2015, tells the story of a client whose Quality Policy didn’t include those exact words but had other words that meant the same thing. Paris asked TC 176 (the ISO committee responsible for ISO 9001) for a ruling whether his client’s Quality Policy was acceptable, and the answer came back No.

Off the cuff I can think of two corporate Quality Policies that are memorable and meaningful.** Neither one has any explicit words about satisfying requirements. Neither one uses the phrase “continual improvement.” But they both define crisply how the company wants to relate to its customers and the world.

• Nordstrom has made the strategic decision that what differentiates them from all their competitors is the level of service they offer. So the first and most important point in their Code of Business Conduct and Ethics is: Use good judgment in all situations. This is the policy that empowers Nordstrom associates to do whatever it takes to make things right for the customer. 
• Lands’ End traditionally advertised a policy that was even shorter and crisper: Guaranteed. Period.*** 

At the other extreme I’ve seen Quality Policies that are six or seven paragraphs long. Maybe it is no surprise that employees of those companies often can't explain those policies very well.

We can do better. But it’s not easy.

__________

* To be clear, auditors are not paid by the finding and no decent auditor wastes your time writing worthless nonconformities just so his report is longer. I’m sorry to say that doesn’t mean I’ve never seen it happen.

** That’s not to say there aren’t others, of course.

*** Just now, as I am writing this post, I checked on the Internet and saw reports that they finally gave up the policy this year (2021). I also found Lands’ End websites (for example this one or this one, as of mid-August 2021) that still use it. So I’m not sure of the current status.

The dark side of process metrics

In my last post I talked at some length about how to define Key Performance Indicators (KPIs) or metrics to measure your processes. But there is a risk here. Too many companies decide that the best way to ensure process-adherence and continual improvement is to give people an incentive, so they tie employee bonus payments to the achievement of specific process metric goals. 

Superficially this idea sounds reasonable. Don’t people work harder when they have incentives? Shouldn’t you pay people more when they do what you want? But in practice it can drive perverse consequences.

Once upon a time I worked in a branch office for a company, where I was responsible for implementing the Quality Management System. It was important to my boss that all of the corporate processes be fully in place and fully adopted, so he gave me a goal – one that counted in my annual bonus – that there should be no Major Nonconformities in the internal audits at that branch. In the abstract, this kind of made sense: a Major Nonconformity would mean that some entire process had never been implemented or had completely broken down, and therefore would suggest strongly that I hadn’t been doing my job. The only problem was that I also did most of the internal audits. So when I encountered a potential Major, what was I supposed to do: overlook it, or torpedo my bonus?

In fact I always wrote them up, and just accepted losing that fraction of the bonus. And I was so inexperienced back then that it didn’t occur to me to explain to my boss that he was (unintentionally) incentivizing me to fake my audits. But when I look back in retrospect I just shake my head in dismay. How could neither of us have seen what a bad idea that was?

More generally, when you incentivize a specific outcome, be prepared for people to increase the frequency of that outcome, even if it becomes disconnected from the overall Quality goal. Once again Scott Adams captured this effect perfectly, in a cartoon strip where the company offers its software engineers a ten-dollar bonus for every bug they find and fix. (Like last time, I am providing only a link and not a copy of the strip in order to stay on the right side of Fair Use.) Nor can you avoid the problem by defining the metric more cleverly, because:

There is no metric in the world that cannot be gamed.

So if bonuses are paid only when the process metrics are green, be prepared for the metrics to turn green regardless how well the processes are doing in real life. This doesn’t have to mean that anyone is actually lying: I assume most people care too much for their integrity to sell it so cheaply. But people are also remarkably creative; so if there is a way to do more of the incentivized activity by disconnecting it from the broader goal, people will figure it out and even think that they are doing the right thing. It’s what you asked for, isn’t it?

You need accurate process metrics to know if the processes are working, so tie your bonuses (if any) to something else.   

     

Updated housekeeping details: let's move to once a week

Back when I started this blog I posted a list of housekeeping details. For the most part, everything I said there still stands.

But I've come to think that posting once every two weeks isn't often enough. So I'm going to shift to once a week -- still on Thursday mornings, at 8:00 am, Pacific Time.

And from time to time there may be unplanned, off-cycle posts like this one, if I want to comment on something right now that isn't a normal part of talking about how QMSes work. Or at any rate I don't want to rule out the possibility.

Once again, comments or feedback are welcome. Thanks!