Do you audit Human Resources?

When you plan out your internal audit program for the year, is the Human Resources department in-scope?

It sounds like a simple question, but it's not. Or at any rate I've found that auditors I respect approach the question very differently from the way I do.

When I first started planning audits, it seemed easy: we were going to audit every department in the company, HR was one of those departments, ergo we audited HR. Simple enough. 

Then one day I was working with a colleague from another division – a fine auditor with decades of experience in the Quality business – who asked me Why? in genuine bafflement. 

His argument was that ISO 9001 is all about customer satisfaction. The whole point of a Quality Management System in the first place is so that you consistently deliver the goods and services that your customers require. So who cares what your hiring process is, or how you do annual reviews? None of those things affect the customer, so they should be out of scope for the audit program.

He did make an exception in case HR was in charge of the company training program. Training shows up under clause 7.2 ("Competence"); so if HR is in charge of training records (and sometimes they are) he was prepared to visit them so he could audit training records and ask about the process for ensuring that all training is up to date. 

But even when you look at the other clauses that discuss how the organization hires or treats its people – see, for example, 7.1.2 ("People") or the Note under 7.1.4 ("Environment for the operation of processes") where there is discussion of social and psychological factors in the work environment – HR's role is usually to do what they are told. The HR Manager isn't going to tell the Operations Manager that he needs another inspector on Line 3. Rather, it's the Operations Manager who determines that an extra inspector is "necessary for the … control of … [the] processes" on Line 3 and who then asks HR to go hire one. So if you want to check how that determination was made (because you are auditing 7.1.2), go ask the Operations Manager and not HR. As for social and psychological factors in the work environment, … when was the last time you saw HR tell Engineering that the product development deadlines have to be more generous in order to avoid burnout in the design team? Well, then.

It's a good argument, and I had to think about it for a while. In the end I think I still disagree, but I've had to build an argument of my own. Here is how I see it now: let me know in the Comments which of us you find more persuasive.

My first point is that ISO 9001 is no longer just about customer satisfaction. It's now about the entire functioning of the organization. When ISO 9000:2015 defines quality* as the "degree to which a set of inherent characteristics of an object fulfils requirements," those requirements can come from any interested party.** Interested parties and their needs are determined way back in clause 4.2, and the list of interested parties should certainly include the employees themselves (among others).

Now, consider an absurd example. Suppose the organization offers its employees health insurance, and there are three plans to choose from: call them Gold, Silver, and Bronze. And suppose further that, no matter how the employees fill out their paperwork, the organization's HR department always signs them up for Bronze!

Clearly in a case like that something has gone wrong in the HR process. It doesn't affect the quality of the goods and services that are delivered to the customer, but it seriously affects the quality of the services provided to the employees. So if I were the auditor I'd want to write a nonconformity against some clause – probably 8.5.1 ("Control of production and service provision"), because I would argue that something was badly wrong with the controls on (internal) service provision.

Naturally a breakdown like this is unlikely. But the point is that it is possible, and that the organization's employees are legitimate interested parties who deserve to have their needs and expectations considered as the organization defines its business processes. Or alternatively, if the organization has deliberately chosen not to treat the employees as interested parties, there should be some documented discussion of that decision in the overall planning for the QMS. And that documented decision itself should then be available to the auditor.

So I have persuaded myself that it is still fair and reasonable to rule HR in-scope for the internal audit program. But have I persuaded you? Leave me some remarks in the Comments to let me know.      

__________

* in entry 3.6.2

** see entry 3.6.4, Note 4

    

What's the difference between PDCA and a Management System?

Last week somebody posted a survey on LinkedIn that asked whether "PDCA (-Plan-Do-Check-Act-) Approach can be referred [to] as [a] Management System." You can find it here. The last time I checked (a few hours before it closed) it had collected 381 votes, of which 78% said Yes and 19% said No.

But this is not a matter of opinion. It's a simple matter of fact, and the correct answer is No. What this means is that nearly 300 people who felt confident answering an Internet poll about management systems actually don't understand what a management system is. How is this possible?

To be clear, I recognize that this is a specialized area. And it's not like I think that if I know something, then everyone should know it. There are lots of topics I know nothing about. But neither do I answer Internet polls about quantum mechanics, or cellular biology, or derivative-pricing in hedge-funds.

I can't answer the question why four-fifths of the people who thought they knew the answer got it wrong. But at least I can explain why the other answer is right.

A management system is like a Constitution: it provides the framework for an organization's rules, and it defines roles and responsibilities. More exactly, a management system defines the processes that you use to do the work of your organization, together with all their interactions and controls. You define the interactions because nobody works in a vacuum: your organization's processes must somehow interrelate or interlock. And you define controls -- of some kind! -- to keep things from going off the rails. Defined roles and responsibilities can be (among other things) a kind of process control; KPIs are another kind. These are all the topics we've been discussing here over the last couple of months.

One way a management system differs from a Constitution is that it is more complete. The Constitution of the United States, for example, tells us how laws are made and enforced but doesn't tell us what the laws themselves actually say. A management system includes not only the framework of processes and authorities, but all the individual procedures and work instructions as well. Every wall poster reminding you to be careful of the risks from your machinery, every company-sponsored training class, and every calendar entry telling you it's time to recalibrate your gauges -- all these are parts of the management system too. 

So if you think of a management system as like a Constitution plus all the subsequent laws and stuff … you won't be too far wrong. Obviously most organizations are a lot smaller than most countries, so their management systems are far smaller and simpler than any country's body of legislation. But if you can think of your company as kind of like a small country, the idea is similar.

PDCA, by contrast, is just a generic model for how to do things -- pretty much anything, in fact. The letters stand for Plan-Do-Check-Act, and you can think of them as a simplified summary of the scientific method. Whatever it is you want to do, start by making a Plan based on what you already know (or think you know); then go Do it; then Check to see if you got the results you expected to get; and finally reAct based on whatever you learned in the "Check" phase -- either by revising your assumptions, updating your procedures, or concluding that everything is perfect and steaming full speed ahead. (Notice, by the way, that the PDCA model is truly generic, while a management system is specific to this or that organization in particular.)

When I say you can use PDCA for anything, I really do mean almost anything. One way or another, most of us use the PDCA model intuitively without thinking about it. Giving it a name just helps formalize the steps so that we can be systematic and intentional about them.

For example, let's say you want to camp out in the woods over summer vacation, and you decide to manage the trip according to the PDCA model:

• During the Plan phase you choose the campsite, decide how many nights to stay, and plan what to take.
• During the Do phase you buy your groceries, load the car, and go camping.
• The Check phase is when you talk about it on the drive home. On the one hand, the mosquitos were a real nuisance. On the other hand, camping next to the lake was great because you could go swimming every morning. And you all agree that next time you've just got to bring a lot more marshmallows to roast over the campfire!
• Finally, during the Act phase, you make some notes of the things you learned this year that you don't want to forget (bug spray, lake, marshmallows) and you paste them on next year's calendar where you will see them next summer.

So of course you can also use the PDCA model when you are implementing and maintaining a management system, in exactly the same way.  

• During the Plan phase you set up all your processes, document them, and appoint people to their respective roles and responsibilities. 
• During the Do phase you go do the regular daily work of your organization. 
• During the Check phase you inspect the results: this might mean looking at your KPIs, or at the results from internal process audits, or something else of the kind. And of course part of inspecting the results is comparing them with your targets. Are the KPIs red or green? Did the audit reveal nonconformities, or not?
• During the Act phase, you make management decisions based on the results you found during the Check phase. If all your KPIs are red, is something wrong with your processes? (Or conversely, if they are all green have you been setting the targets too low?) If the same nonconformities keep showing up in audit after audit, is there some systemic problem that you have yet to find and resolve?
• Then based on the management decisions you made in the Act phase, it's time to Plan for the next period (e.g., for the next year). What KPIs will you measure in this year? Where will you set the target that divides green from red? Also maybe you have had to redefine this or that procedure, based on the systemic problems you uncovered above. All of this goes into the Plan for the next year.
• And then it's time for the next Do phase, once again.
• And so on, over and over.

It's a good approach. It's systematic, and if you apply it right you should learn from your mistakes over time. If you are regularly learning from your mistakes, you should be making the organization better and better. So by all means apply the PDCA model wherever you think it can help you.

But it's just a model, or a method. It can never be a management system by itself.

Have I made the distinction clear? Leave me comments one way or the other.

  

Who needs a Quality Policy?

Most corporate Quality Policies are worthless, and this is one area where the current ISO 9001 standard just makes things worse. But it shouldn’t have to be like this.

The whole point of setting a policy is that it gives you a framework for making decisions, so you don’t have to improvise every case from scratch. Let’s say you sell clothing, and one day a customer comes in to return a sweater. The sweater is made from some expensive fabric and has a label that clearly says “DRY CLEAN ONLY”; but the customer ignored the label, washed it with the rest of his laundry in an ordinary washing machine, and ruined it. So now he wants his money back. If you have a Returns Policy, then your sales associates know what to do: accept the return because keeping customer goodwill is worth putting up with a certain amount of foolishness; or tell the customer he's out of luck because he caused his own problem. No sales associate should have to figure out that answer in real time while the customer is standing there shouting. 

But what kind of decisions are you trying to support with a generalized Quality Policy? Does any employee ever tell himself, “I just can’t decide whether to do a good job or not; so I sure am glad there’s a Policy saying I should”? I didn’t think so.

What’s more, if you don’t know what problem you are trying to solve, you’re not going to solve anything very well. This is why most corporate Quality Policies sound like lukewarm vanilla pudding: they are not written to meet any real need, so they end up as meaningless blather.

This, in turn, enables a well-known auditor trick. Clause 5.2.2 (b) of ISO 9001:2015 requires that “The quality policy shall be communicated, understood and applied within the organization.” So if the audit is going slowly and the auditor wants to pad his list of nonconformities,* he can pull someone off the floor and ask, “What’s the Quality Policy? Explain it to me.” The employee probably went through a training session that covered this back when he was hired, but nobody in the company has ever mentioned it since and the employee has forgotten it. Even if there are posters on the wall spelling out the Policy, the employee can’t explain it because it’s lukewarm vanilla pudding. So the auditor gets to write a nonconformity almost for free. After the audit is done this finding will be assigned to senior management for corrective action; senior management will make everyone go to another training session on the Quality Policy; and a month later everyone will have forgotten it all again. The organization will have gone through lots of activity and will have gotten no meaningful result out of it.

Now, it’s possible to write a memorable Quality Policy. The problem is that ISO 9001 won’t let you. Clauses 5.2.1 (c) and (d) specifically require that your Quality Policy “includes a commitment to satisfy applicable requirements,” and “includes a commitment to continual improvement of the quality management system.” Christopher Paris, in his delightful book Surviving ISO 9001:2015, tells the story of a client whose Quality Policy didn’t include those exact words but had other words that meant the same thing. Paris asked TC 176 (the ISO committee responsible for ISO 9001) for a ruling whether his client’s Quality Policy was acceptable, and the answer came back No.

Off the cuff I can think of two corporate Quality Policies that are memorable and meaningful.** Neither one has any explicit words about satisfying requirements. Neither one uses the phrase “continual improvement.” But they both define crisply how the company wants to relate to its customers and the world.

• Nordstrom has made the strategic decision that what differentiates them from all their competitors is the level of service they offer. So the first and most important point in their Code of Business Conduct and Ethics is: Use good judgment in all situations. This is the policy that empowers Nordstrom associates to do whatever it takes to make things right for the customer. 
• Lands’ End traditionally advertised a policy that was even shorter and crisper: Guaranteed. Period.*** 

At the other extreme I’ve seen Quality Policies that are six or seven paragraphs long. Maybe it is no surprise that employees of those companies often can't explain those policies very well.

We can do better. But it’s not easy.

__________

* To be clear, auditors are not paid by the finding and no decent auditor wastes your time writing worthless nonconformities just so his report is longer. I’m sorry to say that doesn’t mean I’ve never seen it happen.

** That’s not to say there aren’t others, of course.

*** Just now, as I am writing this post, I checked on the Internet and saw reports that they finally gave up the policy this year (2021). I also found Lands’ End websites (for example this one or this one, as of mid-August 2021) that still use it. So I’m not sure of the current status.

The dark side of process metrics

In my last post I talked at some length about how to define Key Performance Indicators (KPIs) or metrics to measure your processes. But there is a risk here. Too many companies decide that the best way to ensure process-adherence and continual improvement is to give people an incentive, so they tie employee bonus payments to the achievement of specific process metric goals. 

Superficially this idea sounds reasonable. Don’t people work harder when they have incentives? Shouldn’t you pay people more when they do what you want? But in practice it can drive perverse consequences.

Once upon a time I worked in a branch office for a company, where I was responsible for implementing the Quality Management System. It was important to my boss that all of the corporate processes be fully in place and fully adopted, so he gave me a goal – one that counted in my annual bonus – that there should be no Major Nonconformities in the internal audits at that branch. In the abstract, this kind of made sense: a Major Nonconformity would mean that some entire process had never been implemented or had completely broken down, and therefore would suggest strongly that I hadn’t been doing my job. The only problem was that I also did most of the internal audits. So when I encountered a potential Major, what was I supposed to do: overlook it, or torpedo my bonus?

In fact I always wrote them up, and just accepted losing that fraction of the bonus. And I was so inexperienced back then that it didn’t occur to me to explain to my boss that he was (unintentionally) incentivizing me to fake my audits. But when I look back in retrospect I just shake my head in dismay. How could neither of us have seen what a bad idea that was?

More generally, when you incentivize a specific outcome, be prepared for people to increase the frequency of that outcome, even if it becomes disconnected from the overall Quality goal. Once again Scott Adams captured this effect perfectly, in a cartoon strip where the company offers its software engineers a ten-dollar bonus for every bug they find and fix. (Like last time, I am providing only a link and not a copy of the strip in order to stay on the right side of Fair Use.) Nor can you avoid the problem by defining the metric more cleverly, because:

There is no metric in the world that cannot be gamed.

So if bonuses are paid only when the process metrics are green, be prepared for the metrics to turn green regardless how well the processes are doing in real life. This doesn’t have to mean that anyone is actually lying: I assume most people care too much for their integrity to sell it so cheaply. But people are also remarkably creative; so if there is a way to do more of the incentivized activity by disconnecting it from the broader goal, people will figure it out and even think that they are doing the right thing. It’s what you asked for, isn’t it?

You need accurate process metrics to know if the processes are working, so tie your bonuses (if any) to something else.   

     

Updated housekeeping details: let's move to once a week

Back when I started this blog I posted a list of housekeeping details. For the most part, everything I said there still stands.

But I've come to think that posting once every two weeks isn't often enough. So I'm going to shift to once a week -- still on Thursday mornings, at 8:00 am, Pacific Time.

And from time to time there may be unplanned, off-cycle posts like this one, if I want to comment on something right now that isn't a normal part of talking about how QMSes work. Or at any rate I don't want to rule out the possibility.

Once again, comments or feedback are welcome. Thanks!

      

Processes 3: Measuring them

When I talk about "measuring" a process, I really mean monitoring or measuring the results. The point is to find out whether your process is succeeding or failing. In other words, Are you getting what you want?

In order to distinguish success from failure, you have to do three things: set up some objective criterion of what constitutes success at the beginning of a cycle (a year, a quarter, a project, or whatever); check at the end of the cycle to see how you did; and then publish the results to anyone who needs to know. I'll talk about each of these.

Define your metric 

Typically you measure each process by one or more performance metrics or Key Performance Indicators (KPIs). First you decide what to measure, and then you set a goal that you want to achieve. 

Maybe you decide to measure the effectiveness of the product development process by setting a target that 80% of your development projects will complete on time and on budget. 

Maybe you decide to measure the effectiveness of the shipping process by setting a target of 98% on-time-delivery of orders. 

In each case the idea is that if the process is working correctly, you will hit the target; and if you fail to hit the target, something might be wrong with the process. In principle you can choose almost anything as long as it makes sense – but let me qualify that remark. Some metric definitions are better than others.

There is an extensive literature about how to design process metrics, and I don't propose to summarize it all. But one of the most common bits of advice recommends defining "SMART metrics," where "SMART" is an acronym defined as follows:

• Specific: Say exactly what has to be done.
• Measurable: Make the goal quantitative, so you can measure whether you got there or how much progress you made.
• Achievable: Don't ask for the impossible, or nobody will bother to try.
• Relevant: Make sure the goal is connected to the work in some natural way.
• Time-bound: Define a goal to be completed by a specific point in the future, so everyone knows what target to aim at.

These are perfectly good criteria, and if you can define your metrics this way by all means do so. But some topics don't lend themselves to the SMART criteria; for example, in some cases the most meaningful measure of success might be qualitative or binary rather than quantitative. So if you find you need a shorter and more general list of criteria for your process metrics, I think the bare minimum list of features for any meaningful metric is this one instead:

• Achievable: Success has to be possible, or no-one will try.
• Non-trivial: Failure has to be possible, or no-one will care.
• Unambiguous: You have to be able to tell the difference between success and failure or your metric does you no good – because the whole point of a metric is to tell you whether you succeeded or failed.

Again, if you are able to define SMART goals, do it. But if you can't, at least make sure your goals are Achievable, Non-trivial, and Unambiguous. Those are – and have to be – the basics.

Measure your performance

This should be straightforward, shouldn't it? You defined your metric so that it's measurable. Go measure it.   😀

Publish your results

Now that you've found out how you are doing, tell the world. 

Not literally the whole world, of course. Don't tell your competitors. Don't post your results on the Internet, or on the front page of the daily newspaper. But inside your organization, transparency is a virtue. 

And really, even if you try to restrict the results to those who have a "need to know," that list is still pretty long. 

• All the functional managers responsible for the process have to be informed, because they can't manage effectively without feedback. 
• All the functional workers who implement the process have to be informed, because you cannot hold people responsible for meeting a goal without telling them what the goal is and whether they met it. 
• Top management have to be informed, because they are steering the whole ship and have to know if they are too close to an iceberg. 

So at that point, why not make the information available to anyone who wants to look it up? Post it on a bulletin board outside your office, and keep it updated as a normal routine. When you are transparent with your colleagues, it says that you trust them; and it makes them more likely to trust you. This is always a good thing. Or think of it this way: 

• If the numbers are good, you have no motive to hide them.
• If the numbers are bad because of reasons outside your control, you have no need to hide them.
• And if the numbers are bad because of reasons inside your control that you haven't addressed yet, you shouldn't be able to hide them. 

Never manage your work in a way that you are ashamed of, and you never have to be ashamed of the results. Even when you fail.

Is that all? Not quite. There are ways that KPIs can go badly wrong, if you aren't careful. But that's a different topic from the one in this post. What I have explained here are the formal criteria for any metrics that are useful and functional. On the other hand, if a KPI meets all the right formal criteria but you are measuring the wrong things, your results still won't be any help. That's the topic of my next post. 

    

Off-cycle post: Can "lying to the public" be part of your process?

This post falls entirely outside my regular, planned sequence (the next of which is going to be Part 3 on process management -- Part 2 is here), and also outside the two-week cadence that I defined back in the beginning. (Is two weeks too long to wait between posts? Leave me a comment either way.) But I want to comment in a timely way on Christopher Paris's recent blogpost as follows:

Caspian Pipeline, Responsible for Falsifying Reports Related to Oil Spill, Holds ISO 14001 Environmental Certifications - Oxebridge Quality Resources

By all means read the whole story for the juicy details, as well as the story from the Moscow Times on which it is based. But the gist is in this one-sentence summary:

The Caspian Pipeline company responsible for falsifying the size of a recent Black Sea oil spill "guaranteed" its environmental safety capabilities by pointing to ISO 14001 certificates issued by Bureau Veritas Group and accredited by UKAS.

Of course the environmental damage is appalling, and the firm's mendacity is morally reprehensible. But I have to admit that when I first read the article, a small contrarian voice in the back of my head asked, "But what if they were following their internal processes? What if one of their processes says, 'In case of an environmental disaster, lie to the press.'? Could that make them still compliant to ISO 14001 after all?"

So I had to go check the standard. Turns out the answer is "No, your process can't tell you to lie to the public," but it's not as obvious as you would hope. The ISO 14001:2015 standard contains 25 instances of the word "communicate" or "communication" in the normative clauses 4-10. Of those 25 instances, three say that communication has to take into account the organization's compliance obligations (7.4.1, 7.4.3, and 9.1.1), and only one (in 7.4.1) requires the organization to "ensure that environmental information communicated is consistent with information generated within the environmental management system, and is reliable." But yes, that should be enough to prohibit lying to the public. Also, if the organization's compliance obligations happen to include truthful reporting then those three references apply as well.

What about other standards? ISO 9001:2015 references "communicate" or "communication" 17 times, but nowhere does it explicitly say those communications have to be truthful. Clause 7.4 gives the most guidance, but all it says is:

The organization shall determine the internal and external communications relevant to the quality management system, including: 
a) on what it will communicate; 
b) when to communicate; 
c) with whom to communicate; 
d) how to communicate 
e) who communicates.

ISO 45001:2018 takes the same approach as ISO 14001, requiring that:

When establishing its communication process(es), the organization shall:
-- take into account its legal requirements and other requirements;
-- ensure that OH&S information to be communicated is consistent with information generated within the OH&S management system, and is reliable.

In other words, standards that regulate topics where people might get hurt and around which there are probably legal regulations say something about keeping your communications "reliable."

Does this incident prove the system of certification and accreditation to be corrupt? Not by itself. In the absence of other information, it's always possible that someone went crazy five minutes before his press conference began, and therefore went completely off-script. No process can prevent isolated instances of non-compliance. What matters is how the organization responds to the situation after the fact, and whether Bureau Veritas follows up this paper trail during their next audit to ensure that the organization's response was meaningful and appropriate. Inquiring minds want to know....

Maybe Christopher Paris will be able to follow up on this news later. If you haven't seen his Oxebridge blog yet, by all means check it out.