Processes 3: Measuring them

When I talk about "measuring" a process, I really mean monitoring or measuring the results. The point is to find out whether your process is succeeding or failing. In other words, Are you getting what you want?

In order to distinguish success from failure, you have to do three things: set up some objective criterion of what constitutes success at the beginning of a cycle (a year, a quarter, a project, or whatever); check at the end of the cycle to see how you did; and then publish the results to anyone who needs to know. I'll talk about each of these.

Define your metric 

Typically you measure each process by one or more performance metrics or Key Performance Indicators (KPIs). First you decide what to measure, and then you set a goal that you want to achieve. 

Maybe you decide to measure the effectiveness of the product development process by setting a target that 80% of your development projects will complete on time and on budget. 

Maybe you decide to measure the effectiveness of the shipping process by setting a target of 98% on-time-delivery of orders. 

In each case the idea is that if the process is working correctly, you will hit the target; and if you fail to hit the target, something might be wrong with the process. In principle you can choose almost anything as long as it makes sense – but let me qualify that remark. Some metric definitions are better than others.

There is an extensive literature about how to design process metrics, and I don't propose to summarize it all. But one of the most common bits of advice recommends defining "SMART metrics," where "SMART" is an acronym defined as follows:

• Specific: Say exactly what has to be done.
• Measurable: Make the goal quantitative, so you can measure whether you got there or how much progress you made.
• Achievable: Don't ask for the impossible, or nobody will bother to try.
• Relevant: Make sure the goal is connected to the work in some natural way.
• Time-bound: Define a goal to be completed by a specific point in the future, so everyone knows what target to aim at.

These are perfectly good criteria, and if you can define your metrics this way by all means do so. But some topics don't lend themselves to the SMART criteria; for example, in some cases the most meaningful measure of success might be qualitative or binary rather than quantitative. So if you find you need a shorter and more general list of criteria for your process metrics, I think the bare minimum list of features for any meaningful metric is this one instead:

• Achievable: Success has to be possible, or no-one will try.
• Non-trivial: Failure has to be possible, or no-one will care.
• Unambiguous: You have to be able to tell the difference between success and failure or your metric does you no good – because the whole point of a metric is to tell you whether you succeeded or failed.

Again, if you are able to define SMART goals, do it. But if you can't, at least make sure your goals are Achievable, Non-trivial, and Unambiguous. Those are – and have to be – the basics.

Measure your performance

This should be straightforward, shouldn't it? You defined your metric so that it's measurable. Go measure it.   😀

Publish your results

Now that you've found out how you are doing, tell the world. 

Not literally the whole world, of course. Don't tell your competitors. Don't post your results on the Internet, or on the front page of the daily newspaper. But inside your organization, transparency is a virtue. 

And really, even if you try to restrict the results to those who have a "need to know," that list is still pretty long. 

• All the functional managers responsible for the process have to be informed, because they can't manage effectively without feedback. 
• All the functional workers who implement the process have to be informed, because you cannot hold people responsible for meeting a goal without telling them what the goal is and whether they met it. 
• Top management have to be informed, because they are steering the whole ship and have to know if they are too close to an iceberg. 

So at that point, why not make the information available to anyone who wants to look it up? Post it on a bulletin board outside your office, and keep it updated as a normal routine. When you are transparent with your colleagues, it says that you trust them; and it makes them more likely to trust you. This is always a good thing. Or think of it this way: 

• If the numbers are good, you have no motive to hide them.
• If the numbers are bad because of reasons outside your control, you have no need to hide them.
• And if the numbers are bad because of reasons inside your control that you haven't addressed yet, you shouldn't be able to hide them. 

Never manage your work in a way that you are ashamed of, and you never have to be ashamed of the results. Even when you fail.

Is that all? Not quite. There are ways that KPIs can go badly wrong, if you aren't careful. But that's a different topic from the one in this post. What I have explained here are the formal criteria for any metrics that are useful and functional. On the other hand, if a KPI meets all the right formal criteria but you are measuring the wrong things, your results still won't be any help. That's the topic of my next post. 

    

Off-cycle post: Can "lying to the public" be part of your process?

This post falls entirely outside my regular, planned sequence (the next of which is going to be Part 3 on process management -- Part 2 is here), and also outside the two-week cadence that I defined back in the beginning. (Is two weeks too long to wait between posts? Leave me a comment either way.) But I want to comment in a timely way on Christopher Paris's recent blogpost as follows:

Caspian Pipeline, Responsible for Falsifying Reports Related to Oil Spill, Holds ISO 14001 Environmental Certifications - Oxebridge Quality Resources

By all means read the whole story for the juicy details, as well as the story from the Moscow Times on which it is based. But the gist is in this one-sentence summary:

The Caspian Pipeline company responsible for falsifying the size of a recent Black Sea oil spill "guaranteed" its environmental safety capabilities by pointing to ISO 14001 certificates issued by Bureau Veritas Group and accredited by UKAS.

Of course the environmental damage is appalling, and the firm's mendacity is morally reprehensible. But I have to admit that when I first read the article, a small contrarian voice in the back of my head asked, "But what if they were following their internal processes? What if one of their processes says, 'In case of an environmental disaster, lie to the press.'? Could that make them still compliant to ISO 14001 after all?"

So I had to go check the standard. Turns out the answer is "No, your process can't tell you to lie to the public," but it's not as obvious as you would hope. The ISO 14001:2015 standard contains 25 instances of the word "communicate" or "communication" in the normative clauses 4-10. Of those 25 instances, three say that communication has to take into account the organization's compliance obligations (7.4.1, 7.4.3, and 9.1.1), and only one (in 7.4.1) requires the organization to "ensure that environmental information communicated is consistent with information generated within the environmental management system, and is reliable." But yes, that should be enough to prohibit lying to the public. Also, if the organization's compliance obligations happen to include truthful reporting then those three references apply as well.

What about other standards? ISO 9001:2015 references "communicate" or "communication" 17 times, but nowhere does it explicitly say those communications have to be truthful. Clause 7.4 gives the most guidance, but all it says is:

The organization shall determine the internal and external communications relevant to the quality management system, including: 
a) on what it will communicate; 
b) when to communicate; 
c) with whom to communicate; 
d) how to communicate 
e) who communicates.

ISO 45001:2018 takes the same approach as ISO 14001, requiring that:

When establishing its communication process(es), the organization shall:
-- take into account its legal requirements and other requirements;
-- ensure that OH&S information to be communicated is consistent with information generated within the OH&S management system, and is reliable.

In other words, standards that regulate topics where people might get hurt and around which there are probably legal regulations say something about keeping your communications "reliable."

Does this incident prove the system of certification and accreditation to be corrupt? Not by itself. In the absence of other information, it's always possible that someone went crazy five minutes before his press conference began, and therefore went completely off-script. No process can prevent isolated instances of non-compliance. What matters is how the organization responds to the situation after the fact, and whether Bureau Veritas follows up this paper trail during their next audit to ensure that the organization's response was meaningful and appropriate. Inquiring minds want to know....

Maybe Christopher Paris will be able to follow up on this news later. If you haven't seen his Oxebridge blog yet, by all means check it out.

    

Processes 2: Owning them

 Along with defining a process, you have to name one person as its Process Owner. To be clear, I don't mean there is only one Process Owner in the whole company: that might happen somewhere, but it's not the normal case. What I mean is that each process has one Owner. Depending how things work at your place, it's possible that one person might end up owning a couple different processes, or possibly not.

The Process Owner is the go-to person for all issues, problems, or discussions related to that process. In more formal terms, the Process Owner is the (one) person who has authority to define the process or to change it, and who also has the responsibility to ensure that it works. Let's unpack this.

First, I say the Process Owner has the authority to define the process (or change it).I never say that he has to have the knowledge or expertise all by himself. 

Of course he might. I once worked at a place where the Warehouse Manager knew the status of every order, every planned shipment, and every piece of inventory; he also knew more about the ins-and-outs of storage and shipment than anyone else in the company. He was awesome.

But it's not always true. I've also seen cases where one process (in this case, the product development process) was so fantastically complicated that there was a whole department devoted to keeping it current. The Process Owner was the overall VP of Engineering, but he had to rely on his experts to discuss the details.

It might happen that one process touches many departments, so that the Process Owner has to get multiple people to buy in before he can finalize his process and publish it. He probably has to align with the owners of the other processes that provide inputs into his, or of those to whom he provides his outputs in turn. All of this is possible.

But at the end of the day, there has to be one person who is responsible. If there isn't one Owner responsible, then in practice nobody is responsible. And in that case, if the process ever fails – or breaks down, or proves dysfunctional (and one day they all do) – nobody is really responsible to fix it. Yes, the organization will find ways to ignore the process or work around it. Work will still get done. But the official goals will all be missed and the formal indicators will all be red. And by then the process will have become just an obstacle, instead of a tool.

Second, I say that the Process Owner has the responsibility to ensure that the process works. This responsibility entails a couple of different things.

On the one hand, when the Process Owner defines the process, he had better insert steps to make sure associates are really following the process and the daily work is on track. These steps might be as simple as filling out a checklist or signing off a traveler, but they could also be a lot more complex if necessary.

On the other hand, the Process Owner also has to check the process performance, to make sure the company is getting the expected results. Then once he has checked the process performance, he has to report to Top Management. If the process is giving the results it is supposed to give, life is roses. If not, he had better find out why not. It might be just a routine blip in performance, a normal statistical deviation that doesn't worry anyone. Or it might be a sign that something is wrong, that the process no longer meets the company's needs and has to be adjusted. Of course you don't want to change your basic processes for light and transient causes; so in the first case it's probably best to sit tight and see if next quarter is better. But no process is perfect, and no process lasts forever. Sooner or later, they all have to be adjusted, reworked, or scrapped. Watch closely so you can tell when this has come to pass.

My third post in this sequence is about how to measure process performance. Reporting to Top Management takes place during Management Review, and I'll discuss that in the future (probably some months from now). 

In other words, this post covers the basic What of the Process Owner's job. In the next post we'll start to discuss the How.