What is Evidence-based decision making?

Last week I quoted the Michael Toy on the unpredictability of major improvements. In the same blog post that I referenced, he also remarks, "Reality Is a Good Thing." His point, of course, is that you have to base your decisions on actual facts, however unpleasant those facts might look. If you base your decisions on wishful thinking instead, that doesn't make the facts go away! All it means is that you will be unprepared when you run smack into them.

This sounds obvious. Why do we even need to mention it?

The problem is that if we are not careful, it is easy for us to confuse ourselves about what the facts really say. It is always tempting to believe what we hear from our friends, or what we think sounds obvious, or what we read in the papers, or what we get from a well-advertised numerical model.

What we need is evidence.

Sherlock Holmes looking for evidence.

ISO 9000:2015 recommends a number of "possible actions" that an organization might choose to take, to ensure that decisions are based on objective facts—that is, on evidence. Among these actions are such measures as:

• determine, measure and monitor key indicators to demonstrate the organization’s performance;
• make all data needed available to the relevant people;
• ensure that data and information are sufficiently accurate, reliable and secure;
• ensure people are competent to analyse and evaluate data as needed. (2.3.6.4)     

These measures should look familiar, because they are all fundamental requirements of the corresponding management system standard ISO 9001:2015. It is only a small exaggeration, in fact, to argue that the whole point of the ISO 9001 schema for quality management systems is to ensure that organizations have the data they need so that they can make evidence-based decisions about how to improve. Look at that list again:

• The organization has to determine key indicators of its own performance: but these are just the quality objectives that ISO 9001 requires in clause 6.2, along with the other performance monitoring required in clause 9.1.
• The data has to be made available to all the relevant people (as I discussed in this post from 2021 about process management).
• The data has to be accurate, reliable, and secure: but this is why the organization has to control its own documentation (as provided in ISO 9001, clause 7.5).
• People have to be competent to understand and analyze the organization's data: but this is why the organization has to determine its competency requirements and then take steps to make sure they are met. (All these requirements can be found in ISO 9001, clause 7.2.)

And so on.

For that matter, what is an internal audit (ISO 9001, clause 9.2) but a tool for collecting evidence—"objective evidence" is the technical phrase—of how well the organization is fulfilling its compliance and quality obligations? Then this evidence is discussed in Management Review (Ibid., clause 9.3) to feed decisions about the strategic direction and tactical implementation of the quality system. Again, the whole structure of the standard's requirements is designed to provide factual evidence to management to serve as the basis for decisions. We discussed a couple of years ago why this information has to be factual, and why (in consequence) you should never lie to your auditor!

So yes, in a sense the principle is obvious. But it's still important to state it explicitly. Fortunately, if you work with a standard like ISO 9001, its requirements do much of the work for you.

Of course, the importance of weighing evidence and accepting reality stretches far beyond the cloistered world of quality management system standards. One of the best stories I ever heard about facing reality head-on comes from cattle ranching, which is a far cry from most applications of ISO 9000 or ISO 9001. It goes like this:

A story is told of a Tonopah preacher who was delivering a funeral sermon over the casket of an old range veteran long and widely known in Nye County. The garrulous clergyman was waxing unctuously eloquent. "Old Dan is not dead," he declared. "He has just taken the highest trail ...." The sardonic voice of another old puncher sounded from a rear pew of the church, "I got a hundred sez he's dead!"*

Reality is a Good Thing.

__________

* Owen Ulph, The Fiddleback: Lore of the Line Camp (San Francisco: Browntrout Publishers, 1995), pp. 69-70.