Integrating risk management with Quality

How does risk management relate to Quality? There's obviously some connection. If you don't think about risks when designing a product, there's a chance it might hurt someone when it goes out to the field. And if a product hurts someone, then it's a bad product—not a quality one. But when you set up a Quality Management System and a product development process, it's not always obvious how to integrate the consideration of risk. Is there some magic spot on the flowchart where you can drop a box called "Risk" and just have done?

Last week I wrote about risk evaluation, taking my lead from a LinkedIn post by Etienne Nichols that focused on the medical device industry (where he works). I was delighted to get a detailed reply from Edwin Bills which clarified some technical points and generously offered guidance to further reading. I wasn't familiar with Bills's work at the time, but I looked around and found that he has had a long and productive technical career in Quality and Risk Management, especially (but not only) with respect to medical devices. He has also published multiple articles. One recent article addresses exactly the question I've asked here.

This article dates from May 2023, not quite two months ago, and is titled "The Intersection Of ISO 13485 And ISO 14971 Under The Proposed FDA QMSR." (You can find it here.) It focuses on the medical device industry, which is not exactly my wheelhouse, so I won't try to comment on the technical details. (Besides, if you are curious about those details you should read the article itself.) But Bills makes a number of broader points that are very valuable.

The first of these points sounds obvious at first, but has deep ramifications: there is a difference between product risks and business risks. In exactly the same way, there is a difference between a standard designed for regulatory compliance and one designed for process management. 

• With respect to the first distinction, product risks are in general a lot more detailed than business risks, and require a much more granular response. Risk analysis at the business level might highlight the need for a new policy or a change in strategy; risk analysis at the product level could identify dozens or hundreds of potential risks, of which let's say twenty or fifty are actionable. Then each of the actionable risks has to be analyzed in detail to determine exactly how to prevent it (or at least mitigate its effects).
• With respect to the second distinction, Bills observes simply that the first edition of ISO 13485 (Quality management systems — Requirements for regulatory purposes) was explicitly based on ISO 9001:1994, because the 20-element structure of the latter was very useful in supporting regulatory compliance. By the time ISO 9001:2015 came out, ISO 13485 no longer copied its structure—because ISO 9001's process approach (together with the frequent use of the phrase "the organization shall determine") made the document altogether too vague to be much help in an FDA audit.

A second general point that Bills makes is that—to answer my question in the first paragraph—there is no magic spot on the flowchart where you can drop a box called "Risk" and just have done. Bills takes considerable pains in his article to trace out the interconnections between ISO 13485 and ISO 14971 (Application of risk management to medical devices). As always, you can find the details in his article (and he provides a map at this link). Here let me say simply that the connections are many and subtle; and while he is clear that you do not have to comply with ISO 14971 in order to comply with ISO 13485, it sure helps. Bills also walks his readers through the entire product life-cycle—from design through development and into production—to show exactly what role risk management has to play at each step of the way.

There is a third point that Bills brings out that is absolutely critical for any risk management methodology. Risk management has to be a living system! It is no good to do your risk analyses and then (as Bills says) simply put the documents in the file and forget about them. In the real world, your experience with the product will continue to grow: every hiccup in production and every customer complaint tell you something you didn't know the day before. And this means you have to pull your risk analyses out of the file, update them with the new information, and check to see how that changes your results. 

For example, maybe some customer finds a way to misuse the product that you never thought of (customers are good at this) and as a result it breaks. (Or, far worse, hurts someone.) Now you know that the product won't withstand that kind of misuse. That's new data, so you have to add it to your risk analysis as an input and then see what changes.

There's a lot more, of course, and if you work in the medical device industry I strongly recommend the article. But these points should apply across all industries, so I am grateful that Bills spelled them out.

           

What makes a risk "acceptable"?

A few weeks ago, Etienne Nichols published an extended post on LinkedIn about risk. Nichols works in the medical device industry, so risk is a critical topic for him. But of course everything we do has some kind of risk—even getting out of bed in the morning! At what point do we decide that the risk is "acceptable" so that we can move ahead?

Unsurprisingly there is a standard that discusses this, and Nichols references it. ISO 14971 specifically covers the "Application of risk management to medical devices," and Nichols quotes it in his long post. First he points out that, with respect to any kind of harm, Risk = Probability x Severity. But what is harm? In the context of medical devices, you'd expect that to mean harm to the patient, and of course it includes that among other things. But it turns out that the actual definition is a lot broader. According to ISO 14971, harm means damage or injury to human health, or damage to property, or damage to the environment. All of these have to be considered in a complete risk analysis. 

Then what makes a risk acceptable? That has to be the benefit which is enabled by the risk, and this too is defined broadly. Nichols quotes the following clarification: "Benefits can include positive impact on clinical outcome, the patient's quality of life, outcomes related to diagnosis, positive impact from diagnostic devices on clinical outcomes, or positive impact on public health." This is a broad list, but it has to be. At the very least it has to be as broad as the range covered by the word harm. What is more, we all know that sometimes the benefits of an action land in a whole different area from its risks. If I drive my car to work, the risks are around things like the morning's traffic or the chance of an accident. But the benefits relate to my job, my coworkers, my customers, and my income. There is no simple arithmetical way to calculate that the benefits outweigh the risks, but a lot of people drive to work every day and are satisfied that it's the right thing to do. There must be some kind of intuitive "calculation" behind that decision, but it's not an arithmetical one.

In the end, Nichols concludes, "No risk is acceptable without the presence of some benefit. But when the stakes are high enough and the benefit is great enough, there's no risk that could be unacceptable.... [W]hen it's all said and done, the acceptability or unacceptability of a risk boils down to the benefit." And this, too, is logical. We all know that in wartime, soldiers risk their lives to take a hill or a position; but if by so doing they can help end the war, that benefit is great enough to make any risk worth it.

There is one way that this balance between risk and benefit can go badly wrong. That's when the parties who face the risk and the parties who decide on the action and reap the benefit are different people. This is called moral hazard, and its consequences are always bad.

For example, let's pretend that I sell a device that doesn't work. [I don't really.] Then I make money but the patients suffer. If I get away with it, what's to stop me doing it again? As long as I don't suffer any consequences, I might deem the patients' risks to be "acceptable."*

In the real world, this is why manufacturers are legally liable for their products—to prevent just that kind of fecklessness by giving the manufacturers some kind of skin in the game. ("Skin in the game" is more or less the conceptual opposite of moral hazard.**) And in the exact same vein, Nichols is careful to point out that ISO 14971 states "...this subclause [ISO14971 A.2.7.4] cannot be used to weigh residual risks against economic advantages or business advantages (i.e. for business decision making)."

It's an important qualification.

But so long as that qualification is understood and in place, it's the benefit that makes the risk acceptable. Nichols makes a good argument for this point.

__________

* Compare also Charlie King's protest song, "Acceptable Risks," which treats the exact same concept in a different context. The refrain runs, "But they told me it was safe, and they swore that was true / They said, The risks are all acceptable—acceptable to who?"  

** Therefore, if you ever find a case where a manufacturer is shielded from legal liability for his products, you should assume that moral hazard is a risk, at least in principle.